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SECTION  I 


BACKGROUND 


INTRODUCTION 

The  objective  of  this  study  was  to  investigate  potential  applications 
of  speech  as  an  information  input/output  (I/O)  mode  between  the  pilot  and 
future  tactical  aircraft  systems.  At  the  inception,  the  project  team  was 
directed  to  assume  that  computer  speech  recognition  and  generation  tech¬ 
nology  would  be  capable  of  operating  in  the  tactical  environment  at  the 
time  the  aircraft  came  into  service.  The  question,  then,  was  how  to  use 
the  technology  effectively. 

The  characteristics  of  “future  tactical  aircraft"  were  divided  into 
two  classifications,  Near  Term  (1980-1990)  and  Far  Term  (1990-2000).  The 
Near  Term  aircraft  study  focused  on  the  F-16A  cockpit  as  the  baseline  de¬ 
sign,  but  was  projected  through  the  F-16C  (MSIP)  configuration  and  the 
F-16E  design.  The  F-16E  may  be  the  most  likely  aircraft  for  the  first 
application  of  speech  1/0  in  the  fighter  cockpit.  The  basis  for  this 
assertion  is  that  the  F-16C  cockpit  design  was  already  "set  in  concrete" 
(February  1982)  and  that,  while  limited  speech  1/0  is  foreseen  as  a  poten¬ 
tial  F-16C  retrofit  program,  the  full  capability  of  speech  I/O  could  only 
be  realized  if  it  were  designed  as  part  of  a  revised  concept  for  cockpit 
information  flow. 

The  speech  1/0  functions  in  the  Far  Term  aircraft  will  be  an  extension 
of  the  F-16E  applications,  although  new  methods  for  integrating  the  speech 
1/0  will  be  likely.  In  the  Near  Term  aircraft,  the  speech  I/O  system  would 
interface  with  other  aircraft  systems  which  use  speech  I/O  functions.  In 
the  Far  Term  aircraft,  speech  I/O  could  be  an  integral  service  system, 
analogous  to  electrical  or  hydraulic  power. 

This  work  expands  previous  studies  of  the  use  of  speech  I/O  to  reduce 
pilot  workload  and/or  provide  new  cockpit  capabilities  (North  and  Lea, 
1981). 

APPROACH 

Three  actions  were  taken  to  identify  and  verify  potential  speech  I/O 
applications  for  fighter  cockpit  design.  First,  we  surveyed  part  of  the 
F-16  pilot  community  to  identify  candidate  tasks  that  could  be  accomplished 
by  speech  I/O.  Second,  we  developed  generic  criteria  for  speech  I/O  task 
verification  which  could  be  applied  to  a  variety  of  future  aircraft  de¬ 
signs.  Third  we  verified  the  selection  of  the  F-16  speech  tasks  using  a 
different  part  of  the  F-16  pilot  community. 

Three  top-level  criteria  were  identified  to  provide  guidance  in  speech 
I/O  task  selection.  These  criteria  were  intended  to  be  applicable  to  both 
the  Near  Term  and  Far  Term  aircraft.  The  criteria  were  that  speech  I/O 
would  have  a  place  in  future  cockpit  designs  only  if:  1)  mission  perfor¬ 
mance  were  improved;  2)  survivability  were  improved;  and  3)  pilots'  skills 
could  be  utilized  moi  a  efficiently. 
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Diverse  professional  opinion  exists  as  to  how  speech  I/O  should  be 
used  in  cockpits  (or  whether  it  should  be  used  at  all).  The  project  team 
believes  that  speech  I/O  is  not  necessarily  the  panacea  for  more  efficient, 
safer  system  design.  For  instance,  speech  I/O  should  not  be  considered  a 
mere  replacement  for  manual  actions,  but  on  the  other  hand,  one  word  could 
replace  many  manual  actions.  Sometimes  it  is  faster  doing  things  manually 
than  by  speech  input,  but  speech  may  help  to  keep  the  pilot's  eyes  out  of 
the  cockpit  and  hands  on  the  flight  controls  at  a  critical  time.  In  summa¬ 
ry,  just  because  a  task  can  be  performed  by  speech  I/O  it  does  not  mean  it 
should  be.  All  speech  I/O  tasks  must  be  considered  in  the  full  mission 
context. 

We  anticipated  that  detailed  analysis  of  cockpit  speech  I/O  would 
uncover  research  issues  not  previously  addressed  by  the  aviation  or  speech 
R&D  communities.  For  example,  although  speech  output  in  the  cockpit  is  a 
natural  way  of  communicating  system  information  to  the  pilot,  it  can  be  a 
source  of  great  annoyance  if  it  competes  with  essential  radio  communicat¬ 
ions.  A  difficult  research  issue,  then,  is  how  to  prioritize  the  radio 
communication  and  system  message  flow  under  all  mission  circumstances. 
Such  research  issues  generally  would  fall  beyond  the  scope  of  this  study. 
However,  they  could  not  be  neglected  because  of  their  potential  negative 
impact  on  the  progressive  development  of  cockpit  speech  applications. 
Consequently,  research  issues  were  identified  during  the  study. 

STUDY  PHASES 

In  the  broader  perspective  of  the  program,  the  study  was  conducted  in 
five  phases  which  are  depicted  in  Figure  1: 

1.  Analyze  Cockpit  Activities  -  In  this  phase  we  studied  the  F-16A 

and  F-16C  (MSI?  projected)  aircraft  systems  through  the  available 

flight  manuals.  We  also  developed  a  mission  scenario  (modified 
European  short  range  interdiction)  which  enabled  us  to  interview  a 
group  of  F-16A  instructor  and  operational  test  and  evaluation 

pilots  to  identify  candidate  cockpit  tasks  for  speech  T/0. 

2.  Speech  I/O  Selection  -  First,  the  candidate  tasks  identified  in  1 
above,  were  sorted  and  classified.  Second,  the  candidate  tasks 
were  examined  with  respect  to  a  set  of  implementation  criteria. 
Third,  potential  speech  I/O  research  issues  were  identified  and 
fourth,  these  were  verified  by  the  pilots  in  F16-A  operational 
squadrons . 

3.  Technical  Feasibility  -  In  this  phase  the  vocabulary  and  syntax 

were  developed  for  the  speech  recognition  systems.  Although  the 
vocabulary  and  syntax  were  developed  around  the  requirements  fore¬ 
seen  for  the  F-16  MS  I P  development  simulation  (see  4),  the  struc¬ 
ture  will  accommodate  generic  groupings  of  tasks.  Engineering 
implementation  criteria  also  were  applied  as  a  dimension  for 
evaluating  technical  feasibility.  The  criteria  were  set  up  to 
select  tasks  for  F-16C  retrofit,  Near  Term  >nd  Far  Term  aircraft 
(based  on  complexity  and  cost). 
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4.  Development  Simulation  Specification  -  A  work  statement  was 
generated  which  delineated  the  requirements  for  a  development 
simulation  to  advance  the  findings  of  this  study.  The  work 
statement  assumed  that  this  work  would  be  done  in  a  pseudo  opera¬ 
tional  environment  (like  an  operational  flight  trainer).  Although 
the  statement  was  directed  at  validating  the  findings  of  the 
study,  we  also  identified  those  research  issues  which  are  amenable 
to  resolution  as  part  of  the  simulation. 

5.  Future  Aircraft  Design  Criteria  -  In  this  final  phase  we  attempted 
to  identify  the  design  criteria  for  the  introduction  of  speech  as 
an  information  input/output  mode  between  the  pilot  and  systems  of 
future  tactical  aircraft  (both  Near  Term  and  Far  Term).  Retrofit 
of  existing  aircraft  configurations  (like  the  F-16C)  was  not 
considered. 

INFORMATION  SOURCES 

Two  sources  of  information  were  available  for  the  study,  tactical 
aircraft  documents  and  subject  matter  experts  (qualified  fighter  pilots). 

Documents 


The  following  documents  were  available: 

1.  F-16A  Flight  Manual  USAF  T.O.  1F-16A-1  and  T.O.  1F-16A-34-1-1 
which  described  the  aircraft  systems  and  their  performance.  This 
information  was  considered  to  be  baseline  systems  data. 

2.  Minutes  of  the  F- 16  C/D  Avionics  System  Design  Review  meeting  in 
February  1982  between  General  Dynamics  and  the  USAF  described  the 
avionic  systems  to  be  used  in  the  F-16C  and  cockpit  configuration. 

3.  Mission  scenarios  for  the  Near  Term  and  Far  Term  aircraft  have 
been  produced  by  both  Rockwell  International  and  General  Dynamics 
Corporation  for  the  USAF.  These  studies  provided  the  basis  for 
developing  a  composite  scenario  to  evaluate  the  viability  of 
cockpit  speech  I/O  in  a  European  short  range  interdiction  setting. 
The  composite  scenario  was  then  used  as  the  focus  of  discussions 
with  the  operational  fighter  pilot  community. 

4.  Automation  in  Combat  Aircraft  cockpits,  the  proceedings  of  an 
industry  workshop  conducted  under  the  auspices  of  the  National 
Research  Council  in  1981. 

Subject  Matter  Experts  (SME) 

SMEs  were  drawn  from  the  fighter  pilot  community  at  different  levels 
of  duties,  experience  in  tactical  aircraft  and  experience  in  the  F-16A 
aircraft.  Pilots  from  Edwards  and  Nellis  Air  Force  Bases  participated  in 
discussion,  structured  interviews,  group  testing  and  general  advise  on  the 
subject  matter.  The  pilots  represented  a  variety  of  duties  with  respect  to 
the  F-16,  including  operational  test  and  evaluation,  tactical  instruction 
and  operational  squadrons. 
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SCENARIO  DEVELOPMENT 


Knowledge  of  cockpit  activities  was  essential  to  enable  selection  of 
speech  I/O  candidate  tasks  and  to  develop  criteria  on  which  the  implementa¬ 
tion  of  these  tasks  should  be  based.  Figure  2  shows  the  relationship  and 
magnitude  of  cockpit  activities  and  the  mission.  The  multiplicity  of 
cockpit  activities  carried  out  by  a  pilot  during  various  missions  was  often 
difficult  for  him  to  describe. 

A  variety  of  scenarios  were  developed  as  focal  points  to  help  extract 
the  cockpit  activity  information.  Six  scenarios  were  examined,  covering 
Near  Term  and  Far  Term  aircraft  operations  in  VFR/IFR  and  Day/Night  condi¬ 
tions.  It  became  clear  that  each  scenario  was  written  at  a  level  too  high 
(general)  to  facilitate  the  extraction  of  cockpit  activity  commensurate 
with  the  analysis  of  speech  I/O  tasks.  Eventually,  one  composite  scenario 
was  developed,  based  on  a  European  short  range  interdiction  in  a  combina¬ 
tion  of  day/night  VFR/IFR  conditions.  The  study  strategy  was  to  analyze 
the  modifying  effect  of  other  scenarios,  such  as  a  night  time  mission  with 
an  advanced  system  like  LANTIRN. 

SPEECH  I/O  CANDIDATE  TASK  SELECTION 

The  task  selection  process  is  shown  in  block  diagram  format  in  Figure 
3.  Candidate  task  identification  was  based  primarily  on  the  outcome  of 
structured  interviews  of  13  pilots  at  Nellis  AFB  during  April  1982.  The 
survey  used  the  composite  scenario  as  the  focus  to  ask  each  pilot  to  evalu¬ 
ate  each  listed  cockpit  task  for  speech  I/O  suitability.  A  total  of  71 
tasks  were  used  in  the  survey  for  the  following  mission  segments: 

o  Cruise  Out 
o  Penetrate 
o  Attack 
o  Egress 

o  Air-to-Air  Engagement 
o  Air  Refueling 

o  Equipment  Failure  (during  approach) 

These  mission  segments  were  chosen  as  being  representati ve  of  periods 
when  cockpit  activities  would  be  high.  During  this  survey  the  pilot  also 
was  asked  to  consider  the  use  of  speech  I/O  in  relationship  to  an  "intel¬ 
ligent"  black  box  equivalent  to  having  a  second  person  in  the  back  seat. 
This  black  box  concept  was  identified  as  a  “Black  Box  Back  Seater,"  re¬ 
ferred  to  as  "83."  Much  additional  information  was  gathered  about  B3 
functions  as  enhancements  to  a  speech  I/O  interface. 

Candidate  task  selection  also  was  influenced  by  the  opinions  expressed 
by  the  16  F - 16  pilots  who  were  interviewed  at  Nellis  and  Edwards  AFBs 
during  the  study.  Over  200  comments  were  recorded  by  the  study  team, 
although  some  were  considered  duplications  or  variations  of  a  common  theme. 
The  process  of  sorting  the  200  comments  provided  much  information  which 
eventually  crystalized  as  candidate  applications  and  research  issues  for 
the  use  of  speech  as  an  information  input/output  mode. 
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Figure  3.  Selection  Process  for  Speech  I/O  Tasks. 


Based  on  the  candidate  task  selection  data,  a  questionnaire  was  devel¬ 
oped  and  distributed  to  pilots  who  had  been  briefed  on  the  study.  This 
questionnaire  solicited  ratings  for  each  speech  application  on  a  scale  of 
1-6  representing  the  usefulness  of  the  application.  Questionnaires  were 
given  only  to  the  nine  pilots  who  had  previously  participated  in  the  study 
because  the  in-depth  interviews  on  speech  I/O  applications  prepared  them  to 
respond  meaningfully. 

A  streamlined  version  of  the  questionnaire  was  administered  to  a 
larger  number  of  F- 16  pilots  in  operational  squadrons  to  validate  the 
candidate  speech  tasks  selected  by  the  initial  sample  at  the  Fighter 
Weapons  School. 

SYNTAX  AND  VOCABULARY  DEVELOPMENT 

One  of  the  most  important  human  interface  issues  in  mechanizing  speech 
I/O  for  airborne  systems  is  the  design  of  the  dialogue  between  the  speech 
recognition  system,  speech  generation  system,  and  the  pilot.  This  process 
consists  of  selecting  the  vocabulary,  governing  the  content  of  the  com¬ 
mands,  and  establishing  the  rules  of  syntax.  In  order  to  minimize  pilot 
training  requirements  and  ensure  pilot  acceptance,  vocabulary  selection 
must  follow  the  common  nomenclature  of  the  particular  aircraft.  A  proposed 
speech  input  and  output  vocabulary  for  tactical  aircraft  speech  application 
is  shown  in  Table  1. 

The  manner  in  which  the  speech  recognition  system  recognizes  single 
words  and  words  strung  together  into  phrases  or  expressions  relies  on  three 
different  levels  of  logical  rules.  The  rules  consist  of: 

o  Front-end  speech  recognizer  rules 

o  Grammatical  rules  (syntax) 

o  Situational  logic  rules. 

These  rules  can  be  described  as  comprising  a  speech  understanding  logic,  as 
depicted  in  Figure  4.  The  majority  of  short  phrases  (e.g. ,  one  or  two 
words)  can  be  handled  directly  by  the  rules  of  speech  pattern  recognition 
provided  by  the  acoustical  front-end  processing  of  the  recognizer. 

Longer  sequences  of  speech  input  are  handled  initially  according  to 
the  same  set  of  acoustical  pattern  recognition  rules.  However,  when  recog¬ 
nition  confusion  or  ambiguity  exists,  a  more  sophisticated  set  of  rules 
must  be  applied.  These  are  known  as  grammatical  rules  which  describe  the 
relationships  among  the  vocabulary  units.  An  example  of  a  set  of  these 
syntactical  rules  which  apply  to  tactical  aircraft  is  shown  in  Table  2  and 
Figure  5. 

The  third  set  of  rules  are  based  on  the  status  of  the  aircraft  sys¬ 
tems,  the  flight  regime  and  mission  phase.  These  provide  the  check  and 
balance  (shown  as  feedback  in  Figure  4)  of  the  accuracy  of  the  speech 
recognition  system.  For  the  safety  of  flight,  and  the  continuing  accep¬ 
tance  of  the  speech  interface  by  the  pilot,  it  is  considered  essential  that 
each  recognition  input  be  checked  against  the  operational  status  of  the 
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Figure  4.  Speech  Understanding  Logic  Relationships. 
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CHANGE  OF  SYSTEM  STATUS  FEEDBACK 


UTTERANCE 


Figure  5.  Depiction  of  Syntax. 


TABLE  2.  PROPOSED  SPEECH  INPUT  SYNTAX 


UTTERANCE  function  [  action  |  expression 

FUNCTION  : :  -  CHAFF  |  FLARES  |  ECU  |  HASTES -ARM  |  LAHTIRN-SEARCB  | 

AIR-TO-GROUND  |  AIR-TO-AIR  |  ...  |  FUEL -REMAIN INC  | 
STORES -REMAINING  |  HEADING-TO-ALTERNATE  !  CLIMB  |  DESCEKT 

■  ACTION  device  +  state  |  verb  +  function 

i  DEVICE  function  |  PANEL  (LIGHTS)  |  POSITION  (LIGHTS)  *  | 

j  NAV (LIGHTS ) 

I 

I  STATE  -  UP  |  DOWN  |  OFF  |  ON  |  MODE  |  RELEASED 

i  VERB  BEGIN  |  RELEASE  j  BEGIN -CHECKLIST  |  EALT -CHECKLIST  | 

|  RES  CM  E-CHECKLIST 

i 

!  expression::*  systee  +  quantity  +  OVER 

j  SYSTEM  ::-  VEF -FREQUENCE  |  VHF -CHANNEL  |  UHF -FREQUENCE  |  UEF-CEANNEL  | 
j  TACAN  |  AIM  !  (LOAD)-D?  |  (LOAD)-FUEL  ]  DISPLAY  | 

|  UPDATE-DP  | 

J  QUANTITY  auaber  [units]  [direction]  [quantity]  |  fomat  [direction] 

!  NUMBER  ZERO  |  ONE  |  TWO  |  THREE  |  FOUR  |  FIVE  |  SIX  |  SEVEN  | 

|  EIGET  |  NINER  |  TEN  |  ELEVEN  |  TWELVE  |  THIRTEEN  | 

FOURTEEN  |  FIFTEEN  |  SIXTEEN  |  SEVENTEEN  |  EIGHTEEN  | 
NINETEEN  |  TWENTY  |  THIRTY  |  FORTY  |  FIFTY  |  SIXTY  | 
SEVENTY  i  EIGHTY  |  NINETY  |  HUNDRED  |  THOUSAND  | 

PLUS  |  MINUS  |  POINT 

DIRECTION  NORTH  |  SOUTH  |  EAST  |  WEST  |  RIGET  |  LEFT  |  MARK  [ 

FORWARD  (TANK)  |  AFT  (TANK) 

UNITS  :  :  -  DEGREES  |  POUNDS  I  ALPHA  I  BRAVO  I  CHARLIE  j  .  .  .  j  ZULU  ] 


FORMAT  ENROUTE  (NAV)  |  TARGET- (POSIT  ION  )  |  CLIMB- (PROFILE )  1 

JT IDS -(UPDATE)  i  STORES -(CONFIGURATION)  |  FENCE- (CHECKLIST) 


SOTE:  Parentheses  indicate  a  specific  example 
generic  category. 


and  brackets 


indicate 
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aircraft  before  it  is  executed.  The  basis  for  such  an  intelligent  speech 
recognition  system  exists  in  the  vast  amount  of  information  that  the 
aircraft  already  "knows"  about  itself.  When  this  information  is  made 
available  to  the  speech  system,  each  speech  recognition  can  be  tested  for 
compliance  with  predefined  situational  parameters  before  the  specific 
command  is  implemented.  This  concept  is  the  origin  of  intelligent  system 
operation.  As  computer  intelligence  capability  grows,  we  can  anticipate  a 
trend  toward  allowing  the  pilot  to  speak  in  free-format  phrases  with  less 
adherence  to  syntactical  rules  dictating  the  order  of  verbs,  nouns,  and 
modifiers.  Until  the  “natural  language  understanding"  capability  is  reali¬ 
ty,  we  must  anticipate  a  need  to  either  (1)  "lockstep"  the  user  through  a 
complex  speech  I/O  transaction  using  prompts  and  menus,  or  (2)  dictate  a 
syntactical  structure  that  must  be  followed  to  execute  the  desired  actions. 
It  seems  clear  that  the  second  alternative  is  preferred  in  the  tactical 
coc kpit. 

VALIDATION  AND  VERIFICATION  OF  SELECTED  SPEECH  I/O  TASKS 

Initial  selection  of  speech  I/O  tasks  was  accomplished  by  a  question¬ 
naire  survey  of  F-16  pilots  at  Fighter  Weapons  School,  which  was  used  to 
select  “candidate"  speech  I/O  tasks.  The  selected  tasks  were  then  subject 
to  a  technical  feasibility  phase  which  judged  whether  they  could  be  engi¬ 
neered  into  the  aircraft  systems.  The  selected  tasks  were  then  subjected 
to  validation/verification  by  a  larger  segment  of  the  population  of  F-16A 
pilots  by  administering  a  shortened  form  of  the  questionnaire  survey  to 
three  operational  squadrons  of  the  474th  Tactical  Fighter  Wing  at  Nellis 
AFB. 

RESEARCH  ISSUE  DEVELOPMENT 

A  total  of  18  research  issues  were  identified  during  this  study.  They 
have  been  classified  into  six  broad  groups:  planning,  role  of  speech  I/O 
in  the  cockpit,  cognitive  limitations,  acoustic  environment,  operational 
environment  and  information  queueing.  The  timing  has  been  identified 
between  the  research  issue  and  the  need  for  the  related  speech  system 
implementation.  This  has  been  done  to  foster  the  generation  of  future 
milestones  for  the  organization  responsible  for  addressing  the  research 
issues.  These  R&D  issues  are  presented  in  Appendix  A. 

An  OFT  or  full  mission  simulator  seems  to  be  the  most  suitable  envi¬ 
ronment  to  resolve  many  of  the  issues  generated  by  the  study. 

SIMULATION  GUIDELINES 

The  analyses  conducted  for  the  present  study  provided  the  formulation 
for  generating  a  framework  for  further  development  of  speech  I/O  in  the 
fighter  cockpit.  This  development  process  was  deemed  most  suitable  for  a 
simulation  environment.  Consequently,  the  results  of  the  present  study 
were  formulated  into  preliminary  guidelines  for  simulation  in  support  of 
speech  I/O  development,  given  in  Appendix  B. 
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SECTION  II 


DETERMINING  AIRCRAFT  SYSTEM  FUNCTIONS 


INTRODUCTION 

A  thorough  evaluation  of  speech  as  a  cockpit  I/O  mode  must  be  based  on 
an  understanding  of:  1)  each  system  function  which  the  pilot  must  perform 
or  respond  to;  2)  the  relationship  among  system  functions  (intra  or  inter 
system). 

Although  the  F-16A  cockpit  was  used  as  the  baseline  for  this  phase  of 
the  study,  the  cockpit  was  already  being  redesigned  in  both  the  Multi  Stage 
Improvement  Program  (MSIP)  and  the  Advanced  Fighter  Technology  Integration 
(AFTI)  programs.  However,  very  similar  functions  must  be  performed  irre¬ 
spective  of  the  F- 16  model  (or  any  other  tactical  aircraft).  In  summary, 
it  was  essential  to  know  what  activities  occurred  in  the  cockpit  and  how 
the  various  aircraft  system  functions  related  to  the  conduct  of  a  mission. 

METHOD  OF  TASK  ASSESSMENT 

Each  F-16A  system  was  studied  using  the  aircraft  flight  manuals. 
(Nuclear  stores  management  and  release  were  excluded).  Each  discrete  event 
such  as  switch  position,  knob  position,  light  on/off  condition,  instrument 
reading,  digital  readout,  was  analyzed  for  type,  functional  content,  human 
engineering  factors,  and  system  interface  design.  Each  task  was  recorded 
on  a  separate  form,  an  example  of  which  is  shown  in  Figure  6.  A  total  of 
37  systems  were  analyzed;  726  system  input/output  tasks  were  identified  and 
documented. 

IDENTIFYING  THE  MORE  COMPLEX  SYSTEMS 

Initially  the  analysis  was  directed  at  understanding  the  aircraft 
systems  and  their  use  during  a  mission.  Subsequent  analysis  of  the  data 
identified  potential  speech  I/O  areas  based  on  their  complexity  as  repre¬ 
sented  by  the  number  of  unique  tasks  in  each  system.  The  results  of  this 
analysis  is  shown  in  Figure  7.  Note  that  the  ten  hatched  areas  account  for 
68%  of  all  discrete  functional  tasks  (The  escape  system  was  considered 
inappropriate  for  speech  I/O).  The  ten  top  ranked  systems  based  on  dis¬ 
crete  functional  task  content  are  also  shown  in  Table  3. 

PILOT/SYSTEM  INTERFACING  DEVICES 

One  outcome  of  the  F- 16  cockpit  function  analysis  was  the  evidence  of 
the  many  types  of  interfacing  devices  with  which  the  pilot  must  contend. 
There  is  no  standard  panel  layout  or  switch.  For  example,  a  spring-loaded 
switch  and  a  push  button  can  both  be  used  to  establish  a  momentary  contact. 
The  shape  of  many  devices  like  knobs  and  levers  are  intentionally  made 
different  to  enable  the  pilot  to  discriminate  among  them  tactually,  while 
flying  "heads  out." 
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Figure  6.  Typical  Completed  Form  Used  in  F-16  Aircraft  System  Functional 
Task  Analysis. 
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figure  7.  F-16A  Cockpit  Functional  Task  Ranking  (N-726)  . 
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How  does  this  situation  apply  to  speech  I/O?  When  a  speech  input  is 
used  to  change  system  status,  feedback  must  be  provided  to  the  pilot.  For 
the  F-16A  and  F-16C  cockpits,  in  many  cases,  this  will  mean  mechanically 
moving  a  toggle  switch,  turning  a  knob  or  rotating  a  thumbwheel  to  a  new 
position.  It  is  a  difficult  design  problem  to  accomplish  these  actions 
automatically  in  the  confinement  of  the  F- 16  cockpit.  We  suspect  that  if  a 
trade-off  is  made,  (which  includes  cost)  the  evidence  will  be  against 
incorporating  the  speech  interfaces  which  involve  changing  the  status  of 
toggle  switches  and  knobs  to  be  compatible  with  changes  in  system  status 
which  are  achieved  by  voice  command. 

For  a  viable  speech  input  interface  in  a  cockpit,  new  types  of 
switches  must  be  integrated  into  the  control  panel.  Touch  switches  and 
push  switches  with  rear-lit  annunciation  are  potential  candidates  to  solve 
the  problem  in  new  cockpit  designs.  Spring-loaded  switches  with  remote 
annunciation  of  system  status  may  provide  a  solution  when  retrofitting 
speech  input  to  existing  "toggle  and  knob"  cockpits. 

SYSTEM  ENGINEERING  CONSIDERATIONS 

Basically  the  F-16A  and  F-16C  aircraft  are  designed  with  inter  and 
intra  system  connection  based  on  analog  principles.  A  powerful  digital  bus 
system  services  the  more  complex  navigation,  fire  control,  HUD,  ECM  and 
threat  warning  systems.  It  is  the  latter  systems  which  are  identified  for 
speech  input/output  and,  from  an  engineering  standpoint,  are  attractive 
interfaces  to  design  because  the  speech  input/output  can  be  another  digital 
signal  on  an  existing  bus  system.  However,  researchers  often  think  that 
just  because  a  digital  bus  exists,  every  piece  of  information  is  available 
continually  on  the  bus.  Not  so,  only  those  parameters  which  need  to  be 
interchanged  between  subsystems  normally  appear  on  the  bus. 

Although  we  did  not  study  the  purpose  of  each  parameter  on  the  F-16A 
and  F-16C  bus  systems,  we  would  be  surprised  if  the  existing  bus  informa¬ 
tion  were  adequate  for  retrofitting  a  limited  speech  I/O  system  including 
an  intelligent  black  box  (B3).  However,  there  may  be  word  address  and 
timing  limitations  for  additional  signals  which  would  be  convenient  for 
speech  I/O. 

The  F - 16  aircraft  system  study  raises  two  issues  in  the  area  ■/  Ji- 
gital  engineering  research.  First,  future  aircraft  which  use  sk**5-*1  I/O 
should  have  a  bus  system  that  is  commens urately  flexible  with  the  demands 
that  the  pilot  and/or  the  related  aircraft  systems  can  place  on  it.  Sec¬ 
ond,  the  redundancy  afforded  speech  I/O  functions  must  be  equal  to  or 
better  than  the  functions  of  the  systems  it  is  controlling.  The  latter 
suggests  distributing  the  speech  I/O  function  around  the  networked  systems 
rather  than  concentrating  it  in  one  “black  box." 
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SCENARIO  DEVELOPMENT  FOR  SPEECH  I/O  ANALYSES 


AVAILABLE  SCENARIOS 

Two  documents  were  provided  by  the  Government  to  provide  mission 
scenario  data.  These  were: 

1.  Air-to-Ground  Cockpit  Design.  Rockwell  International  for  AFFDL , 
1979.  (Secret) 

2.  Flight  Management  for  Ai r-to-Surface  Weapon  Delivery:  Mission 
Scenarios.  General  Dynamics  for  AFWAL/FIGR,  1981.  (Secret) 

The  first  document  was  developed  for  the  Near  Term  timeframe;  the 
second  document  was  developed  for  the  Far  Term  timeframe.  Neither  document 
was  developed  in  sufficient  detail  so  that  it  could  be  directly  applied  to 
the  study.  They  were  written  at  the  mission  event  level  and  did  not  pro¬ 
vide  details  that  could  be  developed  into  a  sequence  of  cockpit  activities. 
Furthermore,  the  far  term  document  was  even  more  lacking  in  detail  because 
critical  events  like  a  surface-to-air  missile  evasive  action  were  developed 
around  aircraft/missile  technology  which  is  nebulous  at  this  time. 

Originally  the  program  proposal  called  for  modifying  the  Near  Term  and 
Far  Term  scenarios  to  make  them  compatible  with  evaluating  cockpit  activi¬ 
ties  related  to  speech  I/O.  After  reviewing  the  Government  documentation, 
it  became  obvious  that  scenario  data  would  have  to  be  developed  rather  than 
modifying  existing  material. 

SCENARIO  DEVELOPMENT 

First,  a  composite  scenario  was  selected  based  on  a  Near  Term  European 
short  range  interdiction  mission.  The  profile  of  this  scenario  is  shown  in 
Figure  3  and  Table  4.  Other  scenario  events,  suspected  to  have  speech  I/O 
applications,  were  added  to  form  a  composite  scenario.  These  events  were 
air-to-air  engagement,  air  refueling  and  an  aircraft  system  failure. 

Second,  an  attempt  was  made  to  expand  the  critical  segments  in  the 
scenario  (i.e.,  cruise  out,  penetrate,  strike  and  egress)  to  achieve  a 
level  of  detail  that  would  be  related  to  cockpit  activities.  It  was  in¬ 
tended  that  the  data  could  be  corrobrated  by  operational  pilots  and  provide 
sufficient  detail  to  elicit  speech  I/O  considerations  for  each  listed  task. 
However,  the  number  of  tasks  became  overwhelming.  At  the  first  rudimentary 
pass  through  the  four  critical  segments,  over  150  activities  were  docu¬ 
mented.  The  use  of  such  a  document  for  eliciting  pilot's  comments  about 
speech  I/O  was  considered  to  be  too  detailed  and  overly  burdensome  for  the 
pilots. 

The  speech  I/O  scenario  was  developed  with  the  assistance  of  senior 
operational  test  and  evaluation  pilots  with  F- 16  experience.  An  example 
from  the  resultant  scenario  is  shown  in  workbook  format  in  Figure  9. 
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Subsequent  evaluation  of  the  scenario  by  instructor  pilots  at  the  Fighter 
Weapons  School  and  the  #422  Test  and  Evaluation  Squadron  (TES)  at  Nellis 
AFB  suggest  that  it  was  at  an  appropriate  level  of  detail  for  the  study. 

The  scenario  included  both  IFR  and  VFR  daylight  segments.  Additional 
scenario  events  to  include  night  operations  with  specialized  equipment 
(LANTIRN)  were  not  developed  in  a  formal  manner.  However,  specific  re¬ 
quirements  arising  out  of  night  operation  events  and  special  systems  were 
included  in  the  data  gathered  at  the  structured  interviews  with  the  pilot 
community. 

Structured  Interviews 


These  interviews  took  place  in  conjunction  with  presenting  the  scenar¬ 
io  work  book  to  individual  pilots.  The  interviews  were  facilitated  by  a 
group  meeting  in  which  a  demonstration  of  the  speech  I/O  concept  was  given. 
The  demonstration  system,  developed  previously  by  Honeywell  SRC  and  the 
Flight  Dynamics  Laboratory,  included  a  stores  management  scenario  executed 
by  computer  graphics  and  an  off-the-shelf  speech  system  driven  by  a  micro¬ 
processor.  The  demonstration  enabled  the  pilots  to  understand  the  poten¬ 
tial  role  of  speech  I/O  by  providing  concrete  examples. 

The  structured  interviews  also  were  used  to  solicit  other  operational 
variations  which  could  modify  the  opinions  expressed  in  the  scenario  work 
books.  These  operational  variations  included: 

1.  Wearing  CBR  gear. 

2.  Use  of  LANTIRN. 

3.  Debriefing  and  briefing  during  short  turn  arounds. 

SCENARIO  SUMMARY 

The  composite  scenario,  which  had  a  tenuous  beginning,  stood  up  well 
to  operational  critique. 

The  available  Far  Term  scenario  material  was  not  sufficiently  robust 
to  facilitate  the  development  of  a  speech  I/O  specific  scenario.  It  is 
likely  that  the  generic  categories  which  have  been  developed  for  the  Near 
Term  aircraft  will  be  applicable  to  the  Far  Term  aircraft.  Thus,  the  need 
for  a  hypothetical  far  term  scenario  is  not  clear  at  this  time. 
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SECTION  IV 


SPEECH  I/O  APPLICATIONS  DEVELOPMENT 


UNDERSTANDING  THE  COCKPIT  OPERATING  PROBLEM 
Pilot  Workload 


The  number  of  cockpit  functions  available  to  the  fighter  pilot  have 
increased  enormously  since  World  War  II.  Advances  in  technology  have 
created  multi-modal  systems,  multi-function  displays  and  a  potential  infor¬ 
mation  overload  for  the  pilot  (Frazier  and  Crombie,  1982).  It  is  no  longer 
simply  a  matter  of  stick  and  throttle,  seat-of-the-pants,  and  nerves  of 
steel.  A  pilot  must  understand  the  mission  objectives  and  the  strategy  and 
tactics  available  to  him.  His  basic  goals  are  to  optimize  mission  success 
and  to  survive.  To  achieve  these  goals  he  must  operate,  manage,  and  con¬ 
trol  a  large  number  of  aircraft  systems. 

Visual  and  manual  channels  are  highly  loaded  for  the  pilot  of  the 
present  day  fighter.  Visual  search  occurs  not  only  outside  the  cockpit,  in 
searching  for  destination  points,  targets,  and  air  traffic,  but  also  within 
the  cockpit,  scanning  the  displays  for  information.  Inside  the  cockpit  the 
information  obtained  visually  consists  of  both  system  status  information  as 
well  as  symbology  that  represents  the  outside  world,  such  as  radar.  The 
pilot  must  visually  monitor  a  large  amount  of  information  to  achieve  the 
desired  enabling  objectives  of  his  mission. 

Although  the  pilot  is  confronted  with  a  vast  array  of  information,  his 
eyes  can  only  perceive  with  good  acuity  in  fovea!  vision  which  subtends 
only  a  few  degrees  of  visual  angle.  Without  being  aware  of  it,  the  pilot 
is  extremely  busy  in  sampling  information  by  eye  movements  during  flight. 
This  relates  to  one  of  the  most  important  characteristics  of  the  highly 
trained  fighter  pilot  which  is  his  control  over  his  own  attention  shifts. 
He  must  know  what  information  is  needed,  when  it  is  needed,  and  where  he 
can  get  it. 

The  pilot  controls  his  aircraft  systems  almost  entirely  by  means  of 
manual  input.  The  manual  input  may  be  discrete,  such  as  button  pushes  or 
rotary  switch  positions,  or  it  may  be  continuous  as  in  stick  and  throttle 
control . 

Speech  I/O  provides  a  method  for  off-loading  the  extremely  busy  chan¬ 
nel  of  eyes/hands.  Asking  for  information  and  receiving  it  verbally  is  a 
very  natural  form  of  human  communication.  It  has  been  used  routinely  in 
two-seat  fighters,  but  it  has  never  been  available  to  the  pilot  of  a  single 
seat  fighter.  Advances  in  technology  now  make  this  form  of  information 
exchange  possible.  Speech  I/O  may  be  a  valuable  asset  to  the  pilot  when  he 
desires  to  input  a  command  or  to  extract  information  from  an  onboard  compu¬ 
ter  while  keeping  his  eyes  out  of  the  cockpit  and/or  while  keeping  his 
hands  on  the  throttle  and  stick  (HOTAS). 
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Eyes  In/Out 

An  eye  movement  requires  200  msec,  on  the  average.  One  cycle  of 
eyes-in,  look  at  a  gauge  or  instrument,  and  eyes  back  out,  probably  takes  a 
minimum  of  one  second.  At  an  air  speed  of  500  knots  this  one  second  trans¬ 
lates  to  a  meaningful  distance,  particularly  at  low  altitude.  In  certain 
situations,  such  as  low  level  flight  or  flying  wing,  the  pilot  may  prefer 
not  to  interact  with  an  onboard  computer  by  looking  down  into  the  cockpit 
at  a  keyboard,  taking  his  hand  off  the  stick  or  throttle,  and  keying-in  the 
information.  Speech  input  allows  the  pilot  an  alternative  method  for  data 
input  while  remaining  eyes-out. 

Wingman/Lead 

The  pilot  of  the  lead  aircraft  has  more  opportunity  to  fly  eyes-in 
than  the  pilot  flying  on  the  wing.  The  wingman  may  wish  to  accomplish  some 
cockpit  functions  while  remaining  HOTAS  and  eyes-out.  This  is  particularly 
true  when  flying  a  tight  formation,  or  when  flying  formation  in  deteri¬ 
orating  weather  or  relatively  low  level.  In  otherwords,  combinations  of 
circumstances  may  induce  a  strong  desire  to  remain  eyes-out  and  HOTAS. 

High  Threat 

Information  about  high  threat  is  of  utmost  interest  to  the  pilot.  It 
has  high  survival  value.  Accurate  and  timely  information  about  the  threat 
type  and  location  would  be  very  useful  to  the  pilot.  Speech  output  pro¬ 
vides  a  possible  way  to  present  this  information.  This  assumes,  however, 
that  the  speech  output  is  driven  by  very  accurate  threat  assessment  infor¬ 
mation.  In  a  high  threat  environment  the  pilot  would  like  to  spend  minimum 
time  with  eyes  in  the  cockpit.  Speech  output  represents  a  valuable  method 
for  presenting  information  to  the  pilot  in  a  high-threat  environment.  The 
HUD,  of  course,  remains  another  medium  for  displaying  important  information 
while  remaining  eyes-out.  The  potential  difficulty  of  using  speech  output 
in  a  high-threat  environment  is  to  avoid  "communications  clutter." 

Communication  Clutter 


Although  the  pilot  receives  most  information  visually,  our  F-16  SMEs 
report  that  a  considerable  amount  of  communications  clutter  occurs  during 
periods  of  heavy  action.  At  that  time  radio  communications  may  include  the 
ground,  FAC,  friendlies,  etc.  The  pilot  must  process  all  of  this  informa¬ 
tion,  often  over  noisy  radios,  and  he  must  integrate  it  within  the  context 
of  the  current  tactical  situation.  Meanwhile,  he  must  fly  his  aircraft, 
accomplish  the  mission,  and  survive.  Speech  output  must  not  interfere  with 
these  tasks.  Speech  output,  however,  can  contribute  to  success  when  it  is 
available  to  provide  information  about  high  threats,  or  other  information 
as  requested  by  the  pilot. 

The  challenge  is  how  to  design  the  audio  world  of  the  cockpit  to 
provide:  1)  alerting  tones;  2)  prioritization  of  messages;  3)  infomation 
presentation;  and  4)  allow  for  pilot  control  over  the  message  queue. 
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uaution/Warning 

There  are  at  least  two  stages  of  information  that  must  be  given  to  the 
pilot  for  caution  or  warnings.  The  first  is  to  get  his  attention.  This 
simply  indicates  to  him  that  there  is  some  problem.  The  second  is  to 
provide  information  about  the  nature  of  the  problem.  This  type  of  infor¬ 
mation  must  be  prioritized  relative  to  other  information  that  the  pilot  Is 
processing  at  the  time.  For  example,  many  caution  messages  would  not  be 
appreciated  by  the  pilot  during  a  high  threat  environment.  There  is, 
therefore,  a  need  for  the  pilot  to  be  able  to  control  the  caution/warning 
information  flow  relative  to  his  current  situation. 

Auditory  tones  can  be  used  for  the  first  function  mentioned  above 
i.e.,  alerting  or  attention  getting.  Speech  output  is  well  suited  to  the 
second  type  of  information  i.e.,  identifying  the  problem.  The  use  of  voice 
warnings  is  being  studied  for  the  F-16  (Davis  and  Stockton,  1982),  and  the 
use  of  tones  and  verbal  information  in  the  cockpit  has  been  studied  by 
Simpson  and  Williams  (1980). 

Information  Flow 


Speech  I/O  capabilities  should  be  added  to  the  fighter  cockpit  only 
through  the  careful  integration  of  visual  and  auditory  information  presen¬ 
tation  to  pilot.  Adequate  feedback  must  be  provided  for  each  pilot  action. 
This  feedback  may  be  either  visual,  auditory,  or  haptic  (by  feel).  An 
analysis  of  the  information  flow  in  the  cockpit  also  must  consider  the  need 
for  the  pilot  to  access  needed  information.  His  information  sources  in¬ 
clude  his  own  memory,  his  knee  board,  all  the  cockpit  instruments,  various 
modes  of  MFDs,  and  radio  communication.  These  sources  can  be  supplemented 
by  an  intelligent  interactive  system  which  enables  the  pilot  to  ask  for  in¬ 
formation  verbally  and  receive  it  through  speech  output  or  visual  display. 

The  temporal  aspects  of  pilot  tasks  are  important  in  the  analyses  of 
cockpit  information  flow.  The  sequences  of  activities  which  the  pilot 
performs,  including  perceptual,  cognitive  and  motor  activities,  must  be 
understood  in  order  to  design  the  information  presentation  optimally.  Such 
analyses  should  occur  at  all  task  levels  in  the  hierarchy,  from  macro,  such 
as  mission  objectives,  to  micro,  such  as  pushing  a  button.  Additionally, 
the  temporal  analysis  of  cockpit  information  flow  must  allow  for  individual 
variation  among  pilots  in  their  inherent  characteristics,  training,  experi¬ 
ence,  and  personal  styles. 

The  impact  of  speech  I/O  in  this  area  occurs  in  several  ways:  infor¬ 
mation  input  can  be  achieved  by  speech;  the  time  required  for  speech  input 
and  recognition  may  be  different  from  manual  input;  and  the  time/location/ 
modality  of  recognition  status  feedback  must  be  considered. 

Similarly,  speech  output  provides  another  avenue  for  information 
presentation  to  the  pilot,  and  the  characteristics  of  speech  output  as  it 
effects  information  flow  and  cockpit  activities  must  be  considered.  Speech 
output  can  be  given  when  a  system  reaches  a  certain  status  (tactical  or 
system  limits,  as  in  a  caution).  Speech  output  can  be  implemented  in 
several  different  ways:  a)  preset/fixed  for  an  aircraft  type;  b)  preset 
before  each  flight  as  part  of  routine  aircraft  preparation;  c)  preset 
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before  flight  by  pilot  preference;  and  d)  requested  during  the  flight  by 
the  pilot.  These  various  implementation  strategies  range  from  extremely 
fixed  to  extremely  flexible  with  respect  to  variation  in  the  information 
flow  and  the  control  allocated  to  the  pilot. 

Anthropometry 

Cockpit  real  estate  is  valuable.  Information  of  high  importance  or 
high  frequency  must  be  readily  available  to  the  pilot.  Such  information 
currently  is  most  likely  to  be  found  on  the  HUD  or  central  and  high  in  the 
cockpit.  The  lowest  priority  information  tends  to  be  available  to  the  side 
and  to  the  rear  of  the  cockpit  where  it  is  difficult  to  see  and  to  reach 
without  twisting  or  turning. 

Speech  I/O  may  prove  beneficial  when  applied  to  cockpit  functions  that 
need  to  be  upgraded  in  terms  of  their  real  estate  allocation,  i.e.,  they 
are  either  more  important  or  more  frequent  than  the  position  to  which  they 
have  been  allocated.  Speech  I/O  allows  access  without  real  estate. 

Sumnary  of  the  Cockpit  Operating  Problem 

Speech  I/O  integration  must  emphasize  the  pilot  as  the  central  limi¬ 
tation  on  information  flow  through  the  fighter  aircraft  system.  Speech  I/O 
must  be  designed  within  the  context  of  the  specific  aircraft  systems  and 
the  mission,  while  maintaining  the  perspective  of  information  flow  through 
the  pilot.  This  analysis  must  include:  a)  sensory  modality  -  vision, 
hearing,  touch;  b)  temporal  requirements  for  detection/perccption,  decision 
making,  and  action;  c)  feedback  -  close  the  loop  around  the  pilot  to  verify 
that  the  desired  state  was  achieved  by  his  action. 

Speech  I/O  provides  a  way  of  putting  into  the  cockpit  a  natural  human 
medium  of  information  exchange,  thereby  off-loading  the  overburdened  eyes 
and  hands  of  the  pilot. 

THE  POTENTIAL  CAPABILITIES  OF  SPEECH  I/O  TECHNOLOGY  IN  THE  TACTICAL  AIR¬ 
CRAFT  COCKPIT 

The  capabilities  and  limitations  of  current  speech  recognition  systems 
place  constraints  on  certain  candidate  applications.  It  is  likely  that  the 
first  airborne  recognition  system  will  be  an  isolated  word,  speaker  depen¬ 
dent  system,  requiring  the  user  to  speak  words  or  short  phrases  of  less 
than  one  to  two  seconds  duration,  and  to  "train  the  system"  prior  to  use. 

A  discrete  word  recognition  (DWR)  system  will  be  quite  sufficient  for 
fighter  cockpit  applications  because  brief  utterances  and  a  relatively 
limited  vocabulary  domain  are  foreseen.  However,  the  requirement  to  pause 
between  utterances  may  be  uncomfortable  for  the  pilot  when  he  wants  to 
rapidly  input  a  string  of  digits.  For  other  applications  (e.g.,  mode 
switching,  quick  activation)  DWR  pausing  constraints  will  not  impact  imple¬ 
mentation  because  one  or  two  word  commands  will  suffice. 


The  second  requirement,  system  training  (also  called  speech  sampling, 
speech  data  collection,  or  enrollment),  should  be  done  on  the  ground  (or 


during  a  training  flight)  by  storing  a  pilot's  speech  data  on  a  miniatur¬ 
ized  storage  device.  Henceforth,  the  system  can  be  initialized  by  placing 
the  pilot's  card  or  chip  in  the  airborne  computer  prior  to  takeoff. 

Speech  Synthesis  and  Digitized  Speech  Generation  (Output) 

Verbal  outputs  can  be  programmed  to  cue  the  pilot  when  an  action  is 
appropriate,  warn  him  of  out-of-tolerance  conditions,  and  act  as  feedback 
and  prompting  when  making  a  transaction  with  the  onboard  computer  (Werko- 
witz,  1980).  There  are  two  categories  of  hardware  techniques  that  are 
currently  used  to  produce  intelligible  speech  sounds  by  machine  --  synthe¬ 
sized  and  digitized  waveforms.  Synthesized  speech  requires  less  computer 
memory,  but  produces  characteristically  metallic,  or  non-human  sounding 
speech.  Digital  speech  is  much  more  natural,  but  requires  mass  storage  of 
digital  waveform  codes.  Digitized  speech  is  created  from  input  by  a  human 
speaker,  and  sounds  very  natural  except  at  very  low  bit  rates.  Compression 
of  digital  speech  using  techniques  such  as  Linear  Predictive  Coding  (LPC) 
look  promising  for  achieving  natural-sounding  speech  with  reduced  storage 
requirements. 

Our  interview  data  have  indicated  that  the  metallic,  unnatural  quality 
of  synthesized  speech  may  actually  be  desirable  to  the  pilot,  because  it 
reduces  potential  confusion  with  normal  radio  communications.  With  either 
type  of  system,  the  computer  will  have  the  capability  to  output  any  length 
or  content  of  verbal  message  to  the  pilot.  Recent  research  has  indicated 
good  response  times  when  pilots  are  presented  with  information  through 
speech  synthesis  (Simpson  and  Williams,  1980). 

APPROACHES  TO  THE  SELECTION  OF  SPEECH  I/O  CANDIDATE  TASKS 

At  the  initial  stages  of  selecting  candidate  speech  I/O  tasks,  a 
decision  must  be  made  as  to  whether  an  objective  or  subjective  approach 
will  be  adopted. 

An  objective  approach,  used  by  North  and  Lea  (1981)  focused  on  using 
detailed  mission  profile  and  timeline  data  in  conjunction  with  a  set  of 
objective  criteria  for  deciding  whether  given  activities  were  suitable 
candidates  for  speech  technology  consideration.  These  objective  criteria 
centered  around  workload  reduction  (visual /manual  task  overload  and  anthro¬ 
pometric  constraints).  In  the  above  study,  objective  criteria  were  used  to 
select  tasks  that  were  then  verified  by  subjective  procedures  using  pilot 
questionnaires. 

In  the  present  study  less  rigid  data  were  available  ( i . e . ,  mission 
profile  and  timelines)  because  of  the  variations  encountered  in  typical 
F-16  sortie  procedures.  Therefore,  the  emphasis  was  shifted  to  obtaining 
candidate  tasks  through  intensive,  structured  subjective  procedures.  A 
mission  profile/timeline  approach  was  used  to  guide  the  subject  matter 
experts  in  their  choice  of  candidate  tasks,  but  no  formal  task-sorting 
procedure  was  used. 

Ultimately,  task  selection  must  progress  to  carefully  controlled 
simulation  tests  to  check  the  validity  of  choices.  These  tests  should  be 
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made  with  pilots,  and  under  realistic  mission  profile  conditions.  Part- 
task  mock-up  simulations  of  the  voice  task  exchanges  may  be  useful  in 
designing  the  best  exchanges  between  pilot  and  system,  but  care  must  be 
exercised  in  extrapolating  results  to  operational  settings.  A  pilot  load¬ 
ing  task  may  be  used  in  these  part  task  simulations  to  increase  time-shar¬ 
ing  realism  (North  and  Mountford,  1981).  A  highly  demanding  manual/visual 
control  task  (e.g.,  compensatory  tracking)  may  be  used  to  obtain  a  higher 
workload  level  representing  flight  control,  for  example. 

CANDIDATE  SPEECH  I/O  TASK  PRESELECTION  PROCESS 

Preselecting  speech  I/O  tasks  based  on  previous  work  on  the  8-52 
(North  and  Lea,  1981)  originally  seemed  to  be  an  attractive  approach  to 
reduce  the  number  of  functional  tasks  involved  (over  700).  Because  of  the 
large  differences  in  tasks  between  the  single-seat  fighter  and  the  manned 
penetration  bomber,  however,  the  present  study  did  not  rely  on  prior  ap¬ 
proaches  to  the  preselection  of  speech  I/O  tasks. 

The  mission  scenario  which  was  eventually  used  for  data  collection  was 
based  on  the  assumption  that  potential  speech  I/O  candidates  would  be 
identified  from  the  conjunction  of  the  aircraft  system  functions  and  the 
scenario  task  demands. 

The  intensive  interviews  conducted  with  pilots  prior  to  handing  out 
the  scenario  workbooks  corroborated  the  validity  of  the  speech  I/O  selec¬ 
tion  methodology.  However,  there  were  cockpit  situations,  especially  those 
involving  emergency  procedures  or  equipment  failure,  which  suggest  that 
speech  I/O  could  be  a  viable  option  for  all  system  functional  tasks  when  a 
degraded  situation  exists.  These  situations  are  difficult  to  cover  ade¬ 
quately  by  a  scenario/workbook  approach.  In  the  present  study,  we  relied 
on  the  intensive  interviews  to  extract  the  candidate  speech  I/O  functions 
in  these  emergency  equipment  failure  situations. 

DATA  COLLECTION  AT  THE  FIGHTER  WEAPONS  SCHOOL  AND  422nd  TES 

Structured  interviews  and  the  completion  of  the  scenario  work  book 
were  conducted  during  April  1982  with  13  pilots  from  the  Fighter  Weapons 
School  and  422  Operational  Test  and  Evaluation  Squadron.  All  pilots  were 
current  in  the  F-16A.  Some  were  familiar  with  the  F-16C  cockpit  configu¬ 
ration  and  later  developments  such  as  MSIP  and  LANTIRN.  There  were  three 
Majors,  one  First  Lieutenent,  and  the  rest  Captains.  Most  of  these  pilots 
had  the  majority  of  their  flight  time  in  the  F-4  aircraft,  and  100-800 
hours  in  the  F-16. 

A  general  briefing  on  speech  I/O  and  a  demonstration  using  laboratory 
equipment  preceded  the  structured  interviews.  The  interviews  were  con¬ 
ducted  with  small  groups  of  pilots,  usually  two  or  three  per  group.  Fol¬ 
lowing  the  interviews,  the  pilots  filled  out  the  scenario  workbook. 

DATA  COLLECTION  AT  EDWARDS  AFB 

Scenario  work  books  were  also  completed  by  three  F-16  test  pilots  from 
the  F-16  Test  Program  and  the  F-16  AFTI  Joint  Test  Force.  These  personnel 
were  included  because  each  had  contributed  subject  matter  in  prior  inter¬ 
views  which  led  to  the  eventual  development  of  the  work  book  scenario. 
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DATA  REDUCTION  AND  IDENTIFICATION  OF  CANDIDATE  SPEECH  I/O  TASKS 

Two  types  of  raw  data  were  reduced;  interview  notes  and  completed 
scenario  workbooks.  The  candidate  speech  tasks  were  selected  from  the 
combination  of  these  data. 

Structured  Interview  Notes 


Over  200  separate  notes  were  produced  from  the  foregoing  interviews. 
This  comment  data  base  represents  the  qualitative  assessment  of  13  highly 
experienced  tactical  pilots  giving  their  opinions  about  how  speech  I/O 
should  be  used  in  future  tactical  aircraft.  In  generating  these  subjective 
opinions  the  pilots  were  weighing  many  factors  such  as  mission  events, 
communications  procedures,  pilot  workload,  physical  access  (anthropometry), 
and  operational  constraints. 

The  data  base  was  divided  into  the  following  generic  categories: 


Speech  Input  Number  of  Items 


Information  Retrieval  Request  15 
Data  Entry  9 
Function  Selection  26 
Quick  Reaction  9 
Operating  Information  Request  3 

Speech  Output 

Abnormality  Information  12 
Warning  Information  3 
Reminder  Service  5 
Memory  Aiding  1 
Tactical  Information  13 
Maneuvering  Critical  Information  6 
State  Change  Feedback  for  Cockpit  Functions  3 


Recorded  comments  also  were  sorted  for  the  following  research  oriented 
categories,  details  of  which  are  shown  in  Appendix  A. 


Research  Issues  Number  of  Items 


Speech  I/O  Introduction  Master  Planning  2 
Role  of  Speech  I/O  in  Cockpit  14 
Cognitive  Limitations  for  Speech  I/O  Mode  5 
Cognitive  Interpretation  in  Speech  I/O  6 
Physiological  Reactions  When  Using  Speech  Mode  3 
Speech  I/O  in  Low/High  Task  Load  Situations  5 
Speech  Output  Information  Priority  2 


This  data  base  became  the  basis  for  the  discussion  of  research  and 
development  issues  presented  in  Appendix  A. 

Note  that  during  the  sorting  process,  no  attempt  was  made  to  remove 
duplications.  Multiple  comments  on  the  same  subject  were  treated  as  cor¬ 
roboration  of  the  importance  of  the  topic. 
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Scenario  Workbooks 


The  completed  workbooks  were  processed  and  the  data  reduced  to  develop 
a  candidate  speech  I/O  listing  based  on  the  scenario.  This  is  shown  in 
Table  5.  The  "NOs"  are  shown  in  Table  6. 

Although  the  scenario  workbooks  made  the  pilots  consider  the  impor¬ 
tance  of  speech  I/O  in  each  phase  of  the  mission,  they  did  not  reflect  the 
wealth  of  diverse  information  gathered  at  the  structured  interviews. 
Therefore,  the  development  of  the  validation  questionnaire  was  weighted  in 
favor  of  the  structured  interview  data  rather  than  the  outcome  of  the 
scenario  workbooks. 

QUESTIONNAIRE  DEVELOPMENT 

A  questionnaire  was  designed  to  allow  F-16  pilots  to  assess  the  utili¬ 
ty  of  speech  I/O  for  a  selected  group  of  candidate  task  applications.  The 
applications  were  compiled  from  the  pilot  interviews  described  above. 
Redundancy  in  pilot  commentary  was  eliminated  to  produce  a  final  set  of  53 
potential  speech  I/O  applications  for  the  F-16.  Each  question  addressed 
certain  attributes  of  the  utility  of  these  applications. 

The  first  of  three  questions  for  each  candidate  application  asked  the 
pilot  to  address  the  utility  of  switching  the  task  or  activity  from  conven¬ 
tional  methods  to  speech  input  (or  output).  This  question  was  answered  by 
marking  a  six  point  scale  anchored  by  the  descriptions  "very  useful"  and 
"never  useful." 

SELECTION  OF  SPEECH  I/O  APPLICATIONS  AND  THEIR  VERIFICATION 

The  results  of  the  questionnaire  provided  the  best  basis  for  selecting 
candidate  speech  I/O  applications. 

The  data  presented  in  the  table  are  more  readily  understood  by  refer¬ 
ence  to  the  scaling  on  the  original  questionnaire,  which  was  as  follows: 


Not  Very 

Useful  Useful 


1 


J _ 1 _ I _ I _ 

2  3  4  5  6 


The  pilots  were  asked  to  circle  only  on  the  cross-hatches,  not  between 
them. 


The  questions  can  be  indexed  to  the  question  numbers  by  referring  to  a 
copy  of  the  questionnaire  given  in  Appendix  C. 

In  terms  of  ranking  the  total  set  of  questions,  the  following  items 
were  rated  most  useful  (mean  rating  over  4.0)  by  the  nine  FWS  instructors: 
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PRIMARY  AIRCRAFT  SYSTEM  CONTROL  SWITCHING 


Content 


Release  Chaff;  Release 
Flares 

Change  TACAN  __  _ _ 

Provide  high  threat 
information  (Type  and 
Di recti  on) 

Activate  ECM 
Select  VHF 
Bogey  informatTon 
Call  for  approach 
plates 

Aircraft  lighting 
control  "kill  exterior 
1 ights," 

At  the  other  end  of  the  scale,  the  speech  applications  that  were  given 
the  lowest  ratings  (Mean  rating  less  than  2.0)  by  the  nine  F-16  pilots  were 
as  to  I  lows: 


Mean  Standard  Speech 


Q# 

Ratings 

Deviation 

I/O 

Content 

14 

1.22 

0.67 

(SO) 

Reports  hung  ordnance 
info 

22 

1.33 

1.00 

(SO) 

Reports  CG  info  for  air 
refuelling 

51 

1.33 

0.71 

(SI/0) 

Ask  for  report  on  bingo 
fuel 

26 

1.67 

1.32 

(SI) 

Ask  for  Air  Terminal 
Information  Service 
( AT I S ) 

12 

1.78 

1.09 

(SO) 

Report  critical  point 
for  pull  out 

13 

1.89 

1.26 

(SO) 

Report  ECM  mode 

Clearly, 

high  threat 

information 

is  rated  as 

a  useful  application,  as 

is  the  capability  to  change  radio  and  TACAN  selections.  The  high  rating  of 
Release  Chaff/Flares"  probably  is  related  to  the  usefulness  of  a  quick 
access  to  these  functions  during  a  high-threat  environment  without  going 
eyes- in  and  taking  the  left  hand  off  the  throttle. 

VALIDATION  SAMPLE 

A  subsequent  sample  of  F-16  pilots  from  the  three  operational  squad¬ 
rons  (428,  429,  430)  of  the  474th  Tactical  Fighter  Wing  at  Nellis  AFB  was 
used  to  expand  and  validate  the  original  sample  from  FWS.  This  was  termed 
the  "OPS"  validation  sample. 

According  to  traditional  validation  techniques,  the  relationship 
between  the  mean  ratings  on  each  question  for  the  FWS  and  the  OPS  samples 
was  calculated.  The  result  was  a  validation  coefficient  of  r  =  .814.  This 


Mean 

Standard 

Speech 

Q# 

Ratings 

Deviation 

I/O 

24 

4.44 

1.81 

(SI) 

34 

4.33 

1.00 

(SI) 

2 

4.33 

1.41 

(SO) 

31 

4.22 

1.56 

(SI) 

33 

4.22 

1.64 

(SI) 

4 

4.11 

1.62 

(SO) 

28 

4.11 

2.09 

(SI) 

45 

4.11 

2.03 

(SI) 

34 


shows  a  strong  relationship  between  the  ratings  of  the  two  groups,  particu¬ 
larly  considering  the  small  size  (N  =  9)  of  the  FWS  sample.  We  can  have 
confidence,  therefore,  in  the  ratings  of  the  combined  samples  with  respect 
to  their  validity  for  speech  1/0  application  in  the  fighter  cockpit. 

QUESTIONNAIRE  RESULTS 

The  questionnaire  ratings  from  the  samples  were  pooled  to  provide  the 
best  data,  based  on  a  sample  size  of  N  =  62.  Table  7  presents  the  data 
from  both  samples.  Rankings  of  the  most  and  least  preferred  applications 
of  speech  input  and  output  were  examined.  See  Appendix  C  for  definition  of 
question  numbers. 

Speech  Input 

Questions  24  -  49  pertained  to  speech  input  applications.  The  26  mean 
ratings  for  speech  input  are  presented  in  rank  order  in  Table  8. 

Again,  it  should  be  noted  that  the  ratings  were  based  on  a  six-point 
scale  ranging  from  1  (Not  Useful)  to  6  (Very  Useful).  There  are  two  ways 
to  consider  the  rating  data  --  relative  ratings  among  the  items  or  absolute 
ratings  anchored  to  the  six-point  scale. 

The  relative  ratings  among  questions  are  shown  adequately  in  the 
previous  table.  The  speech  input  applications  that  received  the  highest 
ratings  tended  to  be  functions  that: 

(1)  were  inconvenient  for  the  pilot  to  accomplish  in  an  eyes-out, 
HOTAS  situation  (e.g.,  high  threat,  low-altitude,  or  wing/ 
weather) , 

(2)  were  awkward  (or  unfamiliar)  to  reach/access  (e.g.,  "Chaff/ 
Flares") , 

(3)  were  discrete  entries,  not  involved  with  flight  control. 

Observation  of  the  mean  ratings  for  the  26  items  on  speech  input  indicates 
a  relatively  uniform  dispersion  of  the  ratings  through  the  top  23  items. 
The  bottom  three  ratings  (Questions  #26,  29,  and  32)  were  noticeably  lower 
than  the  rest. 

When  the  speech  input  ratings  are  viewed  with  respect  to  the  anchor- 
points  of  the  rating  scale,  it  can  be  seen  that  15  items  received  a  mean 
rating  above  4.0,  i.e.,  "useful"  applications.  Furthermore,  if  the  mid¬ 
point  of  the  scale  (approximately  3.5)  is  considered  as  the  division  be¬ 
tween  useful  and  not-useful  applications,  23  of  the  26  items  fall  within 
that  bracket.  The  three  items  mentioned  above  (#26,  29,  32)  were  the  only 
ones  rated  well-below  the  "useful"  half  of  the  scale. 
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TABLE  7.  MEAN  RATINGS  OF  THE  QUESTIONNAIRE  ITEMS  BY  THE  FWS  SAMPLE, 
THE  OPS  SAMPLE,  AND  THE  TWO  SAMPLES  COMBINED 


TABLE  7.  MEAN  RATINGS  OF  THE  QUESTIONNAIRE  ITEMS  BY  THE  FWS  SAMPLE 
THE  OPS  SAMPLE,  AND  THE  TWO  SAMPLES  COMBINED  (Continued) 


Question  No. 

Mean 

FWS 

(N*9) 

SD 

Mean 

OPS 

(N-52) 

SD 

Mean 

Total 
(N=61 ) 

SD 

42 

3.00 

1.87 

4.04 

1.62 

3.88 

1.68 

43 

3.67 

1.94 

4.48 

1.48 

4.36 

1.56 

44 

3.22 

1.99 

3.90 

1.62 

3.80 

1.68 

45 

4.11 

2.03 

4.23 

1.65 

4.21 

1.69 

46 

3.00 

2.24 

4.81 

1.60 

4.54 

1.80 

47 

2.89 

2.15 

3.92 

1.76 

3.77 

1.84 

48 

3.14 

1.57 

4.30 

1.30 

4.14 

1.39 

49 

2.78 

1.99 

3.96 

1.79 

3.79 

1.85 

50 

3.00 

1.94 

3.98 

1.62 

3.86 

1.69 

51 

1.33 

0.71 

3.25 

1.77 

2.97 

1.79 

52 

3.67 

1.80 

4.08 

1.63 

4.02 

1.65 

53 

3.50 

2.27 

3.83 

1.42 

3.78 

1.54 

TABLE  8.  RANK  ORDER  OF  SPEECH  INPUT  APPLICATIONS 
(FWS  ANO  OPS  SAMPLES  COMBINED) 


RANK 

QUESTION  NO. 

MEAN 

1 

33 

5.07 

2 

24 

5.03 

3 

36 

4.82 

4 

34 

4.74 

5 

28 

4.70 

6 

35 

4.58 

7 

46 

4.54 

8 

40 

4.48 

9 

31 

4.41 

10 

43 

4.36 

11.5 

37 

4.25 

12.5 

38 

4.25 

13 

45 

4.21 

14 

48 

4.14 

15 

30 

4.03 

16 

42 

3.88 

17 

44 

3.80 

18 

49 

3.79 

19 

47 

3.77 

20 

39 

3.75 

21 

41 

3.66 

22 

25 

3.53 

23 

27 

3.49 

24 

32 

2.92 

25 

29 

2.85 

26 

26 

2.75 

APPLICATION  IDENTIFICATION 


Select/change  radio  channel s/freqs. 

Deploy  chaff/flares 

Input  NAV  updates  and  other  FCNP  key 

Change  TACAN  channels 

Display  approach  plates 

Input  pre-flight  data 

Call-up  (display  or  report)  checklists 

Request  status  of  aircraft  systems 

Activate  ECM  pod 

Fly  over  mark,  e.g.,  "Mark  Position" 

Mode  selection,  e.g.,  "Select  Air-to-Air" 
Select  radar  coverage  or  radar  display 
Control  interior/exterior  lights 
Control /select  LANTIRN  parameters 
Select  MFD  displays,  e.g.,  "Display 
radar,  left" 

Endurance 

Select  individual  MFD  pages 
Cage/uncage  missile 
Weapon  deployment/control,  e.g.,  "Go 
Slewable" 

Stores  management,  e.g.,  "Select  Master 
Arm" 

Request  "Fence  Check"  items 
Jettison  stores 

Access  JTIDS  info.,  e.g.,  "Display  JTIDS11 
Open/close  air  refuel  door 
Mission  Review 

Access  the  Air  Terminal  Info.  Service 
(ATIS) 


Accordingly,  it  can  be  concluded  that  the  sample  of  62  qualified  F-16 
pilots  rated  88%  of  the  potential  speech  input  applications  as  "useful." 
Because  the  results  of  this  survey  represent  only  one  step  toward  success¬ 
ful  cockpit  application  of  speech  technology,  we  recommend  that  all  of  the 
23  items  be  considered  viable  applications,  worthy  of  further  analysis  in  a 
simulation  environment  (see  Appendix  B). 


Speech  Output 


Questions  1-23  pertained  to  speech  output  applications, 
ratings  for  these  questions  are  given  in  Table  9  in  rank  order. 


The  mean 


TABLE  9.  HIGHEST-RATED  APPLICATIONS  OF 
SPEECH  OUTPUT  IN  RANK  ORDER 


RANK 

QUESTION  NO. 

MEAN 

APPLICATION  IDENTIFICATION 

1 

2 

4.97 

Provide  threat  info.,  e.g.,  "SA-10 
launch,  4  O'clock" 

2 

5 

4.66 

"Bingo  Fuel" 

3 

10 

4.49 

Task  prompts,  e.g.,  "Change  IFF  Code" 

4 

4 

4.46 

Bogey  location,  e.g.,  "Threat  at  2 
O'clock  High" 

5 

19 

4.38 

Bogey  location  (reliability  check  on 

Q  #4) 

6 

1 

4.35 

Ground  proximity  warning  (selectable 
al titude) 

7 

3 

4.30 

New  threats,  e.g.,  "New  Guy,  Quad 

Two" 

8 

17 

4.25 

Master  cautions  announced 

9 

7 

4.20 

Engine  over-temp,  limit 

10 

15 

3.88 

NAY  system  drift 

11 

8 

3.87 

Report  degraded  threat  warning  system 

12 

20 

3.72 

Report  flight  control  battery  low 

13 

18 

3.68 

Report  ECM  status 

14 

6 

3.41 

Reminder  of  Fence  Check 

15 

16 

3.38 

Suggest  options  when  system(s) 
degraded 

16 

12 

3.34 

Solve  and  remind  "pull  out"  point 

17 

23 

3.24 

Warn  of  masking  of  laser-designated 
target 

18 

9 

3.26 

Report  abnormal  fuel  distribution 

19 

21 

3.02 

Spin-direction  announced 

20.5 

13 

3.00 

Mode  changes  confirmed 

20.5 

11 

3.00 

ECM  mode  spoken  upon  request 

22 

14 

2.89 

Hung  ordnance  reported 

23 

22 

1.72 

Reports  CG  change  during  air 
refuel i ng 

It  Is  clear  from  the  data  that  threat  warning/management  information 
was  viewed  by  the  pilots  as  a  useful  application  of  speech  output.  Three 
of  the  top  five  questions  (and  four  of  the  top  seven)  are  related  to  an¬ 
nouncing  threat  information.  This  result  was  corroborated  in  interviews, 
where  the  pilots  emphasized  that  the  one  thing  they  would  like  most  to  hear 
is  high  threat  information  because  of  its  immediate  survival  value. 

Nine  of  the  23  questions  dealing  with  speech  output  were  given  a  mean 
rating  above  4.0,  and  13  questions  were  rated  above  3.5.  This  means  that 
slightly  more  than  half  (57%)  of  the  candidate  speech  output  applications 
were  rated  as  potentially  "useful." 
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It  is  difficult  to  make  an  arbitrary  cut-off  of  the  mean  values  to 
indicate  potential  usefulness  of  the  speech  output  applications.  The  only 
item  that  was  clearly  rejected  was  #22  --  report  CG  change  during  air 
refueling.  Based  on  a  liberal  decision  strategy  to  include  all  items  with 
potential  usefulness,  all  of  the  remaining  22  items  could  be  pursued 
through  simulation.  Adopting  a  more  stringent  decision  strategy,  where 
only  items  with  a  mean  rating  above  4.0  are  accepted,  nine  speech  output 
applications  can  be  recommended  for  further  development  through  simulation. 

It  should  be  noted  that  several  of  the  suggested  applications  of 
speech  output  are  based  on  the  assumption  of  an  "intelligent"  system.  For 
example,  providing  accurate  threat  information  (Q  #2,  4,  19,  and  3)  pre¬ 
supposes  a  system  capable  of  detecting,  classifying,  and  locating  the 
threat.  The  pilots'  ratings  on  the  questionnaire  were  based  on  the  use¬ 
fulness  of  having  a  speech  output  device  provide  the  information.  The 
ratings,  therefore,  reflect  the  value  of  both  the  information  itself  and 
the  method  of  communicating  it  (computer  speech  generation).  Other  items 
which  were  highly  rated,  however,  appear  to  be  straightforward  in  their 
implementation,  such  as  "Bingo  Fuel,"  "Engine  Over-Temp,"  etc. 

Comments  from  the  pilots,  both  in  person  and  written  on  the  question¬ 
naire,  reflected  a  desire  that  speech  output  should  be  used  sparingly  in 
the  fighter  cockpit.  Many  expressed  concern  that  a  "chatty"  computer  would 
become  a  nuisance  in  the  busy  cockpit  environment. 

Speech  Input  and  Output  Compared 

A  comparison  of  the  overall  mean  ratings  of  the  questions  pertaining 
to  speech  input  and  output  are  given  in  Table  10. 


TABLE  10.  OVERALL  MEAN  AND  STANDARD  DEVIATION  OF 
SPEECH  INPUT  AND  SPEECH  OUTPUT  RATINGS 


ITEM 

MEAN 

SD 

Speech  Input 

4.07 

1 .02 

Speech  Output 

3.73 

0.94 

Speech  I/O  Questions 

Although  many  questions  in  both  the  I  and  the  0  portions  of  the  ques¬ 
tionnaire  assumed  or  implied  a  two-way  voice  interaction,  the  four  ques¬ 
tions  (Q  #50  -  53)  that  specifically  addressed  the  linking  of  speech  I  and 
0  received  moderate  ratings,  with  one  exception.  Question  #51  referred  to 
a  verbal  request  for  bingo  fuel,  with  a  verbal  reply  by  the  system.  The 
mean  rating  of  this  question  (2.97)  would  place  it  in  the  less-useful 
category. 
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Generic  Applications  Questions 

Several  additional  questions  were  added  to  the  questionnaire  for  the 
validation  sample.  The  new  questions  were  in  a  different  format,  allowing 
the  pilots  to  provide  general  comments  about  speech  I/O.  Question  54 
requested  the  pilots  to  rate  generic  applications  of  speech  I/O  in  three 
categories  --  Very  Helpful,  Possibly  Helpful,  and  Not  Helpful.  The  results 
of  this  question  are  given  in  Table  11. 

It  is  clear  from  the  data  presented  in  the  table  that  the  F-16  pilots 
considered  speech  I/O  most  helpful  for  threat  management  and  fire  control. 
Not  surprisingly,  speech  I/O  was  considered  least  helpful  for  vehicle 
control.  This  result  is  consistent  with  the  fundamental  principle  that 
speech  I/O  is  not  well-suited  to  continuous  tasks  such  as  manual  flight 
control.  Tasks  involving  discrete  changes  or  data  rates  (throughout)  of 
less  than  about  20  per  minute  are  better-suited  for  speech  I/O. 

One  interpretation  of  the  data  in  Table  11  is  that  for  all  tasks 
except  vehicle  control,  over  90%  of  the  pilots  felt  that  speech  I/O  could 
be  either  "possibly"  or  "very"  helpful. 


TABLE  11.  RATINGS  OF  GENERIC  SPEECH  I/O  APPLICATIONS 

(N  =  50  -  52) 


COCKPIT  FUNCTION 

Very 

Helpful  (%) 

RATING* 

Possibly 
Helpful  (%) 

Not 

Helpful  (“) 

Threat  Management 

70 

28 

2 

Fire  Control 

63 

35 

2 

Navigation 

48 

46 

6 

R/T  Communications 

44 

48 

8 

Weapons  Delivery 

42 

52 

6 

Subsystem  Management 

32 

62 

6 

Target  Acquisition 

29 

67 

4 

Vehicle  Control 

8 

61 

31 

*Data  represent  the  percentage  of  responses  in  each  rating  category. 


Pilots'  Written  Comments 

The  OPS  validation  sample  was  encouraged  to  write  comments  on  the 
questionnaire  whenever  they  wished.  A  transcription  of  those  comments  is 
given  in  Appendix  0. 
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The  authors  wish  to  emphasize  that  these  comments  can  be  very  helpful 
in  the  analysis  of  speech  I/O  design  issues.  The  information  included  in 
Appendix  0  should  be  considered  as  an  important  aspect  of  the  data  col¬ 
lected  in  the  study. 

AGE  HYPOTHESIS 

The  original  sample  of  pilots  at  FWS  was  relatively  uniform  in  compo¬ 
sition,  with  Captains  and  Majors  who  had  the  majority  of  their  flight  time 
in  F-4s ,  before  transitioning  to  the  F-16.  Informal  discussion  with  sever¬ 
al  pilots  led  to  a  question  of  whether  younger  pilots  might  be  more  open  to 
new  technology  than  their  more  experienced  peers.  Consequently,  the  ques¬ 
tionnaire  data  from  both  samples  combined  were  analyzed  for  a  relationship 
between  age  and  mean-rating,  based  on  the  hypothesis  that  higher  ratings  of 
speech  I/O  applications  would  be  given  by  the  younger  pilots,  i.e.,  a 
negative  correlation  was  predicted. 

Results  of  the  analysis  showed  a  very  weak  (r  =  -.106)  and  statis¬ 
tically  nonsignificant  (p  =  .20)  relationship  between  age  and  mean  rating 
on  the  53  questions  for  the  combined  OPS  and  FWS  sample  (N  =  61). 

Several  factors  may  have  contributed  to  the  experimenters'  perception 
that  the  older,  more  experienced  pilots  were  less  accepting  of  the  speech 
I/O  concept:  (1)  senior  people  may  be  more  likely  to  speak  out  about 
system  development  difficulties,  (2)  several  senior  pilots  expressed  great 
satisfaction  with  the  F-16  (relative  to  the  F-4)  and  were  pessimistic  about 
spending  money  on  further  refinement  to  an  excellent  airplane. 

Acceptance  of  New  Technology 

Whether  or  not  age  is  a  factor  in  the  introduction  of  speech  I/O  in 
the  cockpit,  the  acceptance  of  new  technology  by  operational  personnel  can 
be  an  important  issue  (Wylie  and  Mackie,  1982).  Initial  testing  and  de¬ 
velopment  may  most  appropriately  be  done  by  the  more  senior  personnel,  such 
as  instructor  -  or  test-pilots.  The  participation  of  experienced  personnel 
can  contribute  to  the  quality  of  the  applications  development  and  can  ease 
the  transition  of  the  new  technology  by  fostering  acceptance  in  the  opera¬ 
tional  community.  For  these  reasons,  it  is  recommended  that  at  least  some 
of  the  participants  in  the  simulation  development  of  the  speech  I/O  inter¬ 
face  (see  Appendix  B)  be  respected,  experienced  fighter  pilots. 

SPECIAL  CONSIDERATIONS 

The  structured  interviews  were  used  to  solicit  SME  opinion  on  three 
special  speech  I/O  considerations  for  future  tactical  aircraft  operations. 
They  were: 

1.  Wearing  CBR  gear. 

2.  LANTIRN. 

3.  Debriefing  and  briefing  during  short  turn  arounds. 
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CBR  Gear 


Several  pilots  confirmed  that  speech  input  might  be  particularly 
useful  when  wearing  CBR  gear  because  of  the  difficulty  of  pushing  buttons 
while  wearing  bulky  gloves.  The  majority  of  pilots  did  not  expect  a  decre¬ 
ment  in  speech  I/O  system  performance  due  to  CBR  gear  because  the  vocal  and 
auditory  channels  remain  essentially  the  same.  It  is  recommended,  however, 
that  empirical  tests  be  conducted  on  the  effect  of  CBR  gear  on  speech 
recognition  performance. 

LANTIRN 


Pilots  who  had  been  exposed  to  LANTIRN  vigorously  commented  that  the 
speech  I/O  system  requirement  would  be  substantially  different  when  a 
LANTIRN  pod  was  fitted.  Their  opinions  were  based  on  LANTIRN  missions 
being  single  plane,  night  flights  with  little  or  no  communications  with  the 
outside  world.  Therefore,  speech  I/O  could  be  helpful  in  solving  the  mis¬ 
sion  management  problem  which  they  expect  to  experience  on  LANTIRN  types  of 
missions. 

Due  to  security  classification,  the  project  team  remains  unfamiliar 
with  the  nuances  of  LANTIRN  operations.  However,  LANTIRN  should  be  sub¬ 
jected  to  a  separate  speech  I/O  study  during  the  operational  test  and 
evaluation  phase  of  its  development.  Further,  evaluation  should  be  con¬ 
ducted  on  a  LANTIRN  part-task  trainer  or  a  full  mission  simulator  which 
includes  a  simulated  LANTIRN  system. 

Short  Turn-Arounds 


European  operations  for  tactical  aircraft  call  for  short  turn  arounds 
between  missions.  Ouring  this  period  the  aircraft  must  be  refueled  and 
configured  for  the  next  mission.  In  this  same  period  the  pilot  must  de¬ 
brief  on  the  previous  mission  and  obtain  a  briefing  on  the  next. 

The  project  team  was  asked  to  investigate  the  practicality  of  using 
speech  I/O  to  reduce  the  pilot's  on-ground  work  schedule  and  presumably 
reduce  the  turn  around  time. 

If  the  pilot  is  prepared  to  remain  in  the  cockpit  during  turn  around, 
the  use  of  data  linked  information  between  the  operations  center  and  the 
aircraft  can  be  developed  as  a  debriefing/briefing  channel.  ’here  is  no 
doubt  that  speech  I/O  could  contribute  to  this  process.  It  is  conceivable 
that  the  debrief  could  commence  even  before  landing.  Appropriate  questions 
could  be  displayed  on  one  of  the  MFDs,  allowing  the  pilot  to  answer  them 
using  speech  input. 

For  briefing,  the  pilot  could  use  speech  input  to  ask  for  additional 
information  which  is  not  available  from  the  next  mission's  flight  data 
cassette.  However,  such  a  scheme  introduces  larger,  more  complicated 
vocabularies  which  the  speech  recognizer  subsystem  must  accommodate.  It 
also  presupposes  the  availability  of  accurate  and  rapidly  updated  infor¬ 
mation. 
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In  summary,  a  case  can  be  made  for  using  speech  I/O  for  short  turn 
arounds,  including  maintenance  log  entries.  Before  any  development  on 
these  guidelines  is  conducted,  the  time  required  for  pilots  to  be  de¬ 
briefed/  rebriefed  should  be  assessed  and  compared  with  the  normal  time  to 
ready  the  aircraft  for  the  next  mission. 

IMPACT  OF  OF  THE  F-16  AFTI  SPEECH  CONTROL  PROGRAM 

A  benchmark  speech  input  program  is  being  demonstrated  on  the  F-16 
AFTI  aircraft  during  1982/83.  In  the  first  part  of  the  program  a  limited 
speech  recognition  device  manufactaured  by  Lear  Seigler  Inc.  will  be  in¬ 
stalled  to  control  the  sixteen  MFD  mode  select  buttons  which  surround  the 
display.  In  the  second  part  of  the  program  the  Lear  Seigler  unit  will  be 
replaced  by  an  ITT  unit. 

The  program  is  a  benchmark  because  it  represents  the  introduction  into 
an  operating  cockpit  of  speech  input  avionics  built  to  stringent  environ¬ 
mental  specifications. 

The  speech  input  device  is  functionally  limited  to  single  words  which 
control  the  MFD  modes.  To  the  best  of  our  knowledge  there  is  no  speech 
output  capability  in  either  the  LSI  or  ITT  units.  However,  voice  warnings 
have  been  recommended  for  the  F-16  (Davis  and  Stockton,  1982),  and  speech 
output  may  be  included  in  subsequent  phases  of  the  AFTI  program. 

Limiting  the  vocabulary  to  a  small  set  of  control  words  is  a  prudent 
first  step  for  the  harsh  requirements  of  the  figher  cockpit. 

Although  the  program  has  very  visible  benchmark  characteristics,  it 
should  not  be  mistaken  as  a  demonstration  of  what  speech  I/O  and  intelli¬ 
gent  interfaces  will  do  in  the  future. 

Control  of  the  MFD  modes  by  speech  was  an  easy  and  expedient  choice 
because  of  the  digital  data  bus  interface  between  the  MFD  and  the  speech 
input  system.  However,  the  development  of  speech  I/O  functional  applicat¬ 
ions,  especially  those  which  involve  human  engineering  research  issues, 
were  not  the  objective  of  the  program. 

In  sunmary,  the  AFTI  F-16  program  is  quite  valuable  as  an  engineering 
demonstration,  but  too  limited  (and  not  intended)  to  provide  data  on  opti¬ 
mal  applications  of  speech  I/O  in  the  fighter  cockpit  of  the  future. 
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SECTION  V 


SPEECH  APPLICATIONS  IN  NEAR  TERM  AIRCRAFT 


CONCEPT  FOR  SPEECH  I/O  IN  NEW  AIRCRAFT  DESIGN 

We  have  assumed  that  the  Near  Term  aircraft  will  be  more  automated 
than  the  current  tactical  aircraft.  Further,  automation  will  occur  by  the 
Far  Term  aircraft  primarily  in  the  areas  of  subsystem  management,  monitor¬ 
ing,  and  tactics  (which  includes  planning).  Further,  we  have  assumed  that 
a  speech  I/O  interface  will  be  an  integrated  element  of  this  automation  and 
that  it  will  be  interactive  (B3  concept). 

The  basic  automation  architecture  will  be  reflected  in  a  "core"  design 
associated  with  standardized  software  and  hardware  which  is  applicable  to 
more  than  one  aircraft.  This  is  depicted  in  Figure  10. 

The  approach  to  the  speech  I/O  interface  will  not  be  piecemeal;  rather 
it  will  be  in  parallel  with  nearly  all  other  aircraft  systems.  This  design 
will  provide  the  pilot  with  an  effective  alternate  strategy  for  commanding, 
monitoring  and  managing  the  aircraft  of  the  future. 

RATIONALE  FOR  THE  SPEECH  I/O  IN  NEAR  TERM  AIRCRAFT 

There  are  three  main  considerations  which  must  be  carefully  weighed 
before  speech  I/O  is  introduced.  These  are: 

1.  Not  to  introduce  speech  I/O  at  all.  This  decision  is  indicated  if 
no  advantages  have  been  demonstrated ,  in  terms  of  total  system 
performance,  through  the  use  of  speech  I/O. 

2.  Introduce  speech  I/O  for  a  limited  number  of  cockpit  functional 
tasks. 

3.  Introduce  speech  I/O  so  that  it  is  an  alternative  (parallel)  I/O 
mode  which  is  available  to  the  pilot  as  the  task  situation  dic¬ 
tates. 

In  regard  to  the  first  point  above,  even  though  this  study  has  recom¬ 
mended  some  candidate  speech  I/O  tasks,  subsequent  development  simulation 
may  indicate  that  speech  I/O  for  future  tactical  aircraft  cockpit  does  not 
enable  more  efficient  or  safer  system  design.  It  may  simply  add  more 
functions  to  the  cockpit.  Point  2  assumes  that  developmental  simulation 
will  show  that  the  speech  I/O  capability  provides  practical  human  engi¬ 
neering  benefits  for  the  pilot.  Although  this  report  identifies  generic 
categories  of  functional  tasks  which  can  be  conducted  through  a  speecn  I/O 
(eyes  out,  HOTAS,  etc.),  the  selection  of  these  tasks  involved  a  number  of 
subjective  variables,  primarily  the  opinions  of  the  subject  matter  experts 
sampled.  Di  '■  rse  opinion  on  speech  I/O  existed  in  the  pilot  community 
sampled  by  this  study.  This  variability  of  opinion  about  speech  I/O  plus 
the  fact  that  the  benefit  of  speech  I/O  is  situation-specific  supports  the 
notion  of  designing  an  individualized  cockpit  for  future  tactical  aircraft. 
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Part  of  the  individualization  process  would  be  to  provide  speech  I/O 
as  an  alternative  for  nearly  all  functional  tasks  performed  by  the  pilot, 
as  indicated  in  Point.  3,  above.  Tasks  requiring  manual  inputs  could  be 
accomplished  manually  or  by  speech  comnand.  Tasks  requiring  visual  data 
acquisition  by  the  pilot  could  be  accomplished  through  speech  generation. 
Pilots  could  choose  the  most  convenient  mode  for  accomplishing  a  task  in  a 
given  situation.  For  each  task  performed  by  speech  input,  feedback  would 
be  provided  to  the  pilot.  For  each  task  performed  by  speech  I/O  there 
would  have  to  be  a  manual  or  visual  backup.  This  very  generic  but  powerful 
speech  I/O  design  concept  has  been  identified  for  the  purpose  of  this  study 
as  the  "parallel  speech  I/O  concept." 

It  is  obvious  from  the  foregoing  discussion  that  speech  I/O  as  a 
parallel  mode  requires  development  simulation.  One  issue  is  whether  the 
simulation  required  for  points  1,  2,  and  3  should  be  done  in  a  part  task 
environment  or  in  the  full  mission  simulation  context.  As  the  parallel 
speech  I/O  concept,  by  definition,  includes  all  functions  (with  the  excep¬ 
tion  of  maneuvering  the  aircraft)  then  the  simulation  should  be  conducted 
in  the  full  mission  context.  Figure  11  illustrates  a  two  step  approach  to 
a  development  simulation  program  which  will  meet  the  needs  of  Points  1,  2, 
and  3. 

UNCONNECTED  SPEECH  INPUT/OUTPUT  SYSTEMS 

The  Near  Term  aircraft  is  likely  to  be  designed  with  a  speaker  depen¬ 
dent  speech  recognition  subsystem  that  will  have  a  vocabulary  of  control 
expressions  which  probably  will  not  exceed  250  words  and  phrases.  On  the 
basis  of  informed  opinion,  we  estimate  that  recognition  accuracy  must  be 
approximately  99%  in  order  to  achieve  system  effectiveness  a^d  user  accep¬ 
tance.  Each  control  input  will  be  followed  by  feedback  to  the  pilot  con¬ 
sisting  of  a  visible  system  action  or  by  data  displayed  on  the  MFD  and/or  a 
statement  over  the  speech  output  system. 

System  understandi ng  of  the  speech  recognition  subsystem  outputs  will 
be  accomplished  by  a  set  of  syntactical  rules  which  are  dependent  on  mis¬ 
sion  phase  and  system  status.  This  phase  of  system  development  is  probably 
the  largest  and  most  complex  for  the  program.  All  system  functions,  except 
those  which  control  engine  thrust  and  maneuvering  flight,  can  be  achieved 
through  speech  I/O  at  the  discretion  of  the  pilot. 

Speech  output  will  be  integrated  and  prioritized  with  the  aircraft 
communications  and  emergency  audio  warning  systems.  The  pilot  will  be  able 
to  select  the  degree  of  message  priority  he  requires  for  a  specific  mission 
segment.  Waiting  messages,  flagging,  or  message  details  will  be  displayed 
on  a  dedicated  location  of  one  MFD.  An  audio  tone  will  be  used  to  indicate 
the  presence  of  speech  output  messages.  The  speech  generation  subsystem 
will  use  a  digitally  encoded  vocabulary  commensurate  with  the  systems 
status  information.  The  terminology  used  in  the  speech  output  system  will 
be  standardized  and  compatible  with  the  speech  input  system  vocabulary. 
The  naturalness  of  the  voice  used  for  speech  output  is  undecided  at  this 
stage  of  system  development.  Less  than  natural  may  be  beneficial  to  avoid 
confusion  with  radio  communications. 
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AIRCRAFT  DESIGN 


Figure  11.  An  Approach  to  Speech  I/O  Development  Simulation  . 


An  additional  purpose  of  the  speech  recognition  and  generation  subsys¬ 
tem  is  to  facilitate  the  flow  of  information  between  the  aircraft  systems, 
the  pilot,  and  an  intelligent  system  referred  to  earlier  as  B3  (black  box 
back  seater). 

Functional  details  of  the  system  concept  is  shown  in  Figure  12. 
Further  discussion  of  this  concept  is  given  in  Appendix  A  (Cockpit  Speech 
I/O  Research  Issues). 

INTELLIGENT  SPEECH  SYSTEMS  APPLICATIONS 

Intelligent  speech  systems  monitor  system  status  inputs  and  translate 
the  findings  into  speech  output  messages.  Figure  13  depicts  an  intelligent 
system  providing  a  simple  caution  message.  Figure  14  depicts  the  same 
caution  message  said  with  diagnosic  information.  The  intelligent  system 
can  be  programed  to  give  a  warning  and  remedial  action,  the  example  given 
in  Figure  15  assumes  that  the  aircraft  is  in  the  uncontrollable  flight 
regime  and  that  a  specific  action  should  be  taken  to  rectify  the  situation. 

INTERACTIVE  SPEECH  LOOPS 

Figure  16  depicts  a  speech  input/output  interface  with  no  interconnec¬ 
tion  between  the  I/O  system  elements.  Examples  of  generic  speech  output 
applications  have  been  given  in  the  previous  paragraph.  Simple  examples  of 
speech  input  control  of  systems  are  given  in  Figures  17  and  18. 

The  speech  input  and  output  interfaces  can  be  made  to  interact  in  a 
powerful  combination  by  closing  a  speech  loop  around  the  system  elements. 
Figure  19  depicts  this  interactive  concept.  Inputs  from  aircraft  systems 
are  analyzed  by  the  intelligent  system  which  triggers  a  speech  output 
message  and  at  the  same  time  asks  the  pilot  what  action  should  be  taken. 
Examples  of  this  concept  are  is  shown  in  Figures  20  and  21. 

In  summary,  the  speech  I/O  is  a  natural  interface  between  the  aircraft 
and  the  pilot.  This  interface  can  be  enhanced  when  it  is  combined  with  a 
intelligent  system. 

RELATIONSHIP  TO  SPEECH  I/O  RETROFIT 

For  the  purposes  of  this  study.  Near  Term  aircraft  precludes  retrofit 
of  speech  I/O  equipment  to  such  aircraft  as  the  F- 16  A/B  and  F- 16  C/D 
configurations.  The  Near  Term  aircraft  is  considered  to  be  a  vehicle  that 
has  its  genesis  in  an  aircraft  like  the  F-16E. 

As  mentioned  in  Section  III  of  this  report,  the  speech  I/O  retrofit  to 
existing  or  inmediate  production  aircraft  is  a  difficult  problem.  Some 
reasons  for  this  are  as  follows: 

o  Feedback  to  the  pilot  by  altering  the  position  of  toggle  and  knob 
switches  for  input  commands 

o  Indication  of  spring  loaded  toggle  or  push  switch  operation  for 
input  commands 
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Figure  12.  Speech  Interactive  System  Functional  Design  for  Tactical  Aircraft. 
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igure  13.  An  Example  of  an  "Intelligent"  Caution  Message. 


Figure  15.  Non  Standard  Warning  with  Recommendations 


Figure  19.  Generic  System  Design  for  an  Interactive  Speech  Loop  ■ 
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o  Suitable  display  and  control  of  speech  output  messages  to  the  pilot 

o  Installation  and  aircraft  wiring  complexity  of  prol i ferati ng  digi¬ 
tal  speech  I/O  intefaces  with  analog  aircraft  systems 

o  The  potential  requirement  for  speech  I/O  system  redundancy 

o  Potential  parameter  signal  addressing  and  timing  limitations  on 
existing  digital  bus  systems. 

All  of  these  problems  have  solutions,  providing  time  and  money  are  avail¬ 
able. 


The  near  term  speech  I/O  design  is  conceived  as  transcending  the 
foregoing  retrofit  problems  by  innovative  design  of  the  cockpit  and  its 
system  controls  during  the  aircraft  detailed  design  phase.  Ideally,  we 
.elieve  that  the  speech  I/O  for  the  Near  Term  aircraft  should  be  conceived 
as  an  alternate  input/output  control  interface  between  the  pilot  and  the 
aircraft.  In  suninary,  if  the  pilot  wants  to  use  speech  to  convey  manual 
actions  or  visual  information  then  he  has  the  freedom  to  do  so.  However,  a 
generic  system  concept  of  this  type  requires  a  basic  speech  I/O  interface 
design  technology  to  be  developed. 

IMPACT  OF  THE  PARALLEL  SPEECH  I/O  CONCEPT  FOR  NEW  AIRCRAFT  DESIGN 

Two  major  considerations  are  involved  for  Near  Term  aircraft  design. 
These  are: 

1.  The  effect  the  parallel  speech  I/O  concept  will  have  on  the  cock¬ 
pit  design. 

2.  The  effect  it  will  hive  on  the  aircraft  subsystem  design. 

Cockpit  Design 

The  cockpit  design  will  require  an  entirely  new  approach  to  be  taken 
to  the  design  of  switches,  knobs,  and  levers  which  occupy  much  of  today's 
aircraft  cockpit  space.  Touch  switches  with  back  lit  annunication  will 
replace  many  of  the  toggles,  knobs  and  buttons  found  in  present  aircraft. 
Current  advances  in  touch-panel  and  membrane  switches  are  compatible  with 
the  digital  requirements  for  speech  I/O  implementation.  Likewise,  advanced 
displays  such  as  HUD  and  MFDs  will  be  compatible  with  speech  I/O. 

For  each  manual  action  a  speech  input  expression  will  be  available. 
The  manual  or  speech  input  will  be  accompanined  by  a  feedback  message  to 
the  pilot,  either  visual  or  aural.  For  each  discrete  signal  from  the 
aircraft  subsystems  there  will  be  a  visual  indication  and/or  a  speech 
message  provided  to  the  pilot.  The  visual  indication  may  be  shown  on  a 
master  panel,  on  an  MFD/HUD,  or  on  an  individual  switch.  The  speech  mes¬ 
sage  may  occur  either  without  pilot  control  or  tne  pilot  can  be  informed 
visually  or  aurally  that  a  message  is  waiting  for  his  attention. 
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For  a  new  aircraft  that  will  enter  service  before  the  later  part  of 
the  1980  decade,  aircraft  subsystem  design  is  expected  to  be  similar  to 
current  configurations,  i-e.,  the  utility  systems  (electrical,  hydraulic, 
etc-),  will  have  analog  controls.  The  avionics  will  be  more  closely  inte¬ 
grated  into  a  total  system  concept  with  control  by  dig' t a  1  signals.  How¬ 

ever,  each  system  will  continue  to  have  a  black  box  or  control  unit  which 
is  unique  to  its  function. 

In  this  dedicated  box/system  environment,  the  speech  subsystem  will 
interface  with  all  other  aircraft  systems.  A  high  degree  of  system  iso¬ 
lation  will  have  to  be  maintained  in  the  speech  unit  because  of  potential 

corrmon  mode  failure.  Failure  probability  will  have  to  be  exteremly  low. 

The  speech  system  will  interface  digitally  with  the  avionics  system  and 
through  digital  analog  conversion  with  all  other  subsystems  for  system 
isolation.  It  is  probably  prudent  to  design  the  A/D  and  D/A  interfaces  as 
part  of  each  parent  subsystem. 

The  intelligent  characteristics  of  the  speech  subsystem  will  depend  on 
the  desires  of  the  individual  pilot  and  will  be  mission  dependent.  These 
characteri sties  will  be  programmable  for  each  mission  through  the  data 
transfer  unit  (DTU)  which  is  "loaded"  into  the  aircraft  before  each  flight. 

In  summary,  the  Near  Term  aircraft  design  can  capitalize  on  the  emerg¬ 
ing  display  and  speech  I/O  technologies  to  provide: 

o  An  alternate  interface  between  the  pilot  and  the  aircraft  subsys¬ 
tems. 

o  A  speech  system  which  will  provide  speech  commands,  warnings, 
caution,  diagnostics,  recommendations,  calculations,  information 
retrieval,  checklists  and  debriefing  facilities. 

o  Individualized  outputs  and  priorities  to  suit  individual  pilots  and 
mission  requirements. 


SECTION  VI 


SPEECH  APPLICATIONS  IN  FAR  TERM  AIRCRAFT 


COCKPIT  MAN-MACHINE  INTERFACE  CONCEPT 

During  the  study  we  experienced  difficulties  in  obtaining  information 
on  the  mission  roles  which  discriminate  between  Near  Term  and  Far  Term 
aircraft.  The  primary  source  of  information  on  which  differences  in  speech 
I/O  requirements  could  be  formulated  was  the  "Automation  in  Combat  Air¬ 
craft"  published  by  the  Comnittee  on  Automation  in  Combat  Aircraft,  Air 
Force  Studies  Board,  Assembly  of  Engineering,  National  Research  Council, 
1982.  This  study  was  directed  to  examine  automation  issues  in  combat 
aircraft  for  1980  and  beyond. 

Fundamentally,  the  speech  I/O  concept  described  as  "the  parallel 
speech  I/O  concept"  as  discussed  in  Section  V  of  this  report,  is  not 
expected  to  differ  for  the  Far  Term  aircraft  cockpit.  However,  it  is 
hypothesized  that  the  aircraft  subsystem  interface  with  the  speech  system 
will  change  radically.  A  methodology  for  the  progressive  development  of 
the  parallel  speech  I/O  concept  for  the  far  term  aircraft  is  shown  in 
Figure  22. 

FAR  TERM  AIRCRAFT  IMPLICATIONS 

The  Far  Term  aircraft  is  expected  to  include  new  technology,  more 
systems,  more  operating  modes,  and  more  automation  to  assist  the  pilot  in 
managing  this  prol iteration  of  cockpit  functions.  Although  future  system 
designs  were  not  available  for  the  preparation  of  this  report,  some  of  the 
potential  technologies  may  be  surmised  from  popular  media,  such  as,  more 
guided  weapons,  increased  ECM,  1 aser-tracki ng  weapons,  automatic  target 
recognition,  automatic  terrain-following,  reduced  radar  return  ("stealth"), 
increased  communications  capability  (including  satellite),  and  increased 
speed  and  range.  In  a  general  sense,  this  means  that  more  things  will  be 
happening  to  the  pilot  in  less  time.  The  need  for  rapid  and  accurate 
communication  between  the  pilot  and  his  automated  systems  will  become 
essential  to  avoid  overloading  the  pilot.  Speech  I/O  can  play  an  important 
role  by  providing  an  alternative  to  eyes-hands  as  an  interface  between  the 
pilot  and  his  systems. 

Changes  in  speech  technology  are  certain  to  occur  during  the  same 
timeframe.  Connected  word  recognition  (CWR)  exists  currently  in  a  small 
minority  of  systems.  This  capability  can  be  expected  to  advance  and  there¬ 
by  avoid  the  bothersome  pauses  between  vocabulary  items.  At  a  minimum,  the 
capability  for  recognizing  connected  digits  should  be  provided  to  facili¬ 
tate  input  of  NAV  updates,  radio  frequencies,  etc. 

Recognition  systems  will  be  capable  of  larger  vocabularies  as  they 
advance  in  development.  This  change  may  not  prove  essential  for  cockpit 
applications,  since  current  recognizers  already  are  capable  of  vocabularies 
up  to  several  hundred  or  even  a  thousand  words.  Overly  large  vocabularies 
might  prove  detrimental  if  they  represent  an  increased  memory  load  on  the 
pilot  (see  Appendix  A). 
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yure  L'i.  Progressive  Development  of  the  "Parallel"  Speech  I/O  Concept. 


Because  the  pilot  of  the  Far  Term  aircraft  will  be  able  to  accomplish 
nearly  all  cockpit  functions  by  voice,  if  he  chooses,  the  need  for  secure- 
speech  will  arise.  The  capability  of  others  to  manipulate  cockpit  func¬ 
tions  by  interjecting  simulated  speech  waveforms  must  be  prevented.  Simi¬ 
larly,  an  expansion  of  the  speech  I/O  system  could  include  secure  coded 
speech  communication. 

The  combination  of  AI  and  speech  I/O  will  be  the  area  of  most  poten¬ 
tial  for  the  Far  Term  aircraft.  The  "intelligent"  systems  will  have  impli¬ 
cations  beyond  assistance  in  correct  speech  recognition.  They  should  be 
capable  of  supporting  the  pilot  in  tasks  such  as  target  detection/iden¬ 
tification,  subsystems  selection/management,  fault  diagnosis,  and  even 
assessing  the  probabilities  for  alternative  tactics.  Speecn  I/O  would  be  a 
natural  channel  of  communication  between  the  pilot  and  his  intelligent 
system. 


It  will  not  be  surprising  if  speech  I/O  takes  its  place  alongside 
eyes/hands  as  a  primary  pi lot/ai rcraft  interface  in  the  Far  Term  fighter 
aircraft. 

SPEECH  I/O  INTERFACE  DESIGN 

Increased  automation  in  the  Far  Term  aircraft  will  place  the  pilot  in 
more  of  a  tactical  management  role  in  which  the  emphasis  will  be  on  closer 
integration  of  the  various  system  sensors  and  computers.  Distributed 
processing  of  the  majority  of  subsystem  functions  is  a  natural  outcome  of 
the  integration  design.  Most  aircraft  subsystem  actuation  will  be  by 
digital  signal  using  common  digital  bus  distribution.  System  redundancy 
will  be  inherent  in  the  distributed  processing  concept.  Each  distributed 
processor  can  be  expected  to  have  its  own  speech  I/O  to  interface  with  the 
pilot  for  the  functions  performed  by  that  processor.  The  concept  is  illus¬ 
trated  in  Figure  23.  Speech  system  backup  is  inherent  in  the  design  con¬ 
cept.  If  one  speech  processor  fails,  its  companion  unit  on  the  bus  system 
wi 11  take  over. 

During  the  next  15  years,  improvement  in  speech  recognition  technology 
will  begin  to  eliminate  current  constraints  such  as  speaker  dependency  and 
restrictive  vocabularies.  Consequently,  in  the  Far  Term  aircraft  the 
speech  system  will  be  more  flexible  and  more  reliable. 
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SECTION  VII 


SPEECH  INPUT/OUTPUT  IMPLEMENTATION  RECOMMENDATIONS 


CONSIDERATIONS 

The  study  team  believes  that  future  tactical  aircraft  cockpits  will 
become  more  automated.  Automation  will  change,  and  hopefully  reduce  pilot 
workload.  Mission  effectiveness  will  be  increased  because  it  will  allow 
the  pilot  to  direct  his  attention  to  critical  actions. 

Speech  I/O  is  one  tool  available  to  cockpit  designers  to  achieve  a 
compatible  interface  between  the  pilot  and  the  automated  systems.  However, 
there  is  no  systematic,  widely  applied  technology  available  for  allocating 
functions  between  automated  systems  and  the  pilot.  Furthermore,  there  are 
no  criteria  which  can  be  applied  to  weigh  the  cost  of  automating  a  par¬ 
ticular  cockpit  function  against  the  resulting  improvement  in  mission 
performance. 

However,  while  more  restrictive  task  selection  may  be  required  for 
speech  I/O  retrofit  of  existing  aircraft,  a  strong  case  can  be  made  to 
implement  the  "parallel  speech  I/O  concept"  in  Near  Term  and  Far  Term 
tactical  aircraft.  The  "parallel"  concept  (see  Section  V)  will  provide  the 
flexibility  required  for  mission  and  tactical  strategy  development  and 
perhaps,  more  importantly,  suit  the  individual  needs  of  overloaded  combat 
pilot  in  fulfilling  his  mission.  Speech  I/O  provides  a  parallel  mode  to 
manual  and  visual  communications  for  man-machine  interaction. 

In  reviewing  current  cockpit  speech  I/O  implementation  development 
programs,  the  following  issues  should  be  considered: 

AFTI-16  program  is  a  benchmark  program  to  demonstrate  the  feasibility 
of  speech  recognition  technology  in  a  harsh  environment.  It  is  not 
necessarily  representative  of  the  best  applications  of  speech  I/O  for 
future  tactical  aircraft  cockpits. 

F-16C  (MSIP)  program  is  too  far  along  in  cockpit  design  for  a  speech 
I/O  interface  to  be  considered  for  the  production  aircraft.  If  it  is 
decided  to  introduce  speech  I/O  into  the  F-16C  on  a  retrofit  basis, 
there  is  a  potential  danger  to  the  future  of  all  speech  I/O,  because  a 
hasty  retrofit  installation  may  be  "turned-off1  more  than  it  is  used. 
To  overcome  human  engineering  problems,  speech  I/O  interfaces  appear 
to  require  very  systematic  development  and  integration  with  complex 
functional  systems. 

Airframe  Prime  In-House  Development  programs  are  likely  to  produce 
different  approaches  to  implementing  speech  I/O  interface  design.  No 
doubt  each  product  will  be  serviceable  because  the  airframe  primes 
have  the  simulation  resources  which  are  necessary  to  develop  satis¬ 
factory  systems.  It  would  appear  that  some  form  of  a  standardized 
approach  to  cockpit  speech  I/O  development  is  warranted.  Obviously 
this  would  best  be  accomplished  by  the  Wright-Patterson  Aeronautical 
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Laboratories  or  by  independent  organizations  with  no  vested  hardware 

i nterests. 

An  earlier  figure  entitled  “An  Approach  to  Speech  I/O  Development 
Simulation"  has  been  repeated  as  Figure  24  on  the  next  page  to  reinforce 
the  reconmended  methods  for  systematic  development.  The  approach  uses  part 
task  simulation  to  evaluate  the  effectiveness  of  selected  speech  candiaates 
which  are  an  output  of  this  study.  The  next  step  is  to  investigate  the 
efficiency  of  the  parallel  concept  on  a  full  mission  simulator  using  a  wide 
cross  section  of  the  tactical  pilot  community. 

If  the  Near  Term  development  program  is  successful,  then  the  Far  Term 
aircraft  program  will  be  a  natural  evolution.  If  the  Near  Term  develop¬ 
ment  program  is  not  technically  feasible  or  is  unacceptable  to  the  pilot 
community,  the  whole  subject  should  be  shelved  pending  technology  improve¬ 
ment  or  the  changing  needs  of  the  operational  community. 

In  summary,  implementation  hinges  on  the  rigorous  evaluation  of  the 
two  basic  concepts,  restricted  versus  parallel  implementation.  Such  an 
evaluation  should  be  conducted  in  light  of  the  research  issues  shown  in 
Appendix  A. 

Speech  I/O  Interface  Functional  Specification 

Any  speech  I/O  interface  implementation  on  a  simulator  or  aircraft 
should  meet  a  standard  set  of  functional  requi rements.  An  example  of  such 
requirements  is  presented  in  Appendix  B  and  is  titled  "Research  Features 
and  Design  Requirements  for  Speech  Interfaced  Tactical  Aircraft  Simula¬ 
tion."  Although  the  specification  is  directed  at  the  simulation  which  is 
the  outcome  of  this  project,  it  is  structured  in  a  generic  format  which 
should  make  it  usable  on  any  other  simul ation/ai rcraft  speech  I/O  endeavor. 
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CANCEL  COCKPIT 
SPEECH  I/O  PROGRAM 


RELEASE  TASKS  FOR 
RESTRICTED  SPEECH 
I/O  APPLICATIONS  IN 
AIRCRAFT  MAJOR 
UPDATING  PROGRAMS 


RELEASE  TASKS  FOR 
RESTRICTED  SPEECH 
I/C  APPLICATIONS  IN 
NEW  AIRCRAFT  DESIGN 


USE  MANUAL  AND  SPEECH 
I/C  CONCEPT  FOR  NEW 


AIRCRAFT  DESIGN 


Figure  24.  An  Approach  to  Sceecn  I/C  Development  Simulation. 
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SECTION  VIII 


SUMMARY  AND  CONCLUSIONS 


The  present  study  included  a  total  sample  of  slightly  over  60  pilots 
currently  flying  the  F-16A.  Of  this  sample,  the  vast  majority  (>90%)  felt 
that  speech  I/O  could  be  helpful  in  the  cockpit  for  certain  functions  and 
under  certain  circumstances.  A  list  of  over  200  potential  applications  was 
generated  from  a  scenario  workbook  and  intensive  interviews  with  13  in¬ 
structors  or  test  pilots  at  Nellis  AFB.  This  list  was  culled  to  approxi¬ 
mately  50  potential  applications,  largely  by  eliminating  duplicate  or 
similar  suggested  applications. 

Speech  input  was  rated  as  most  useful  for  selecting  radio  channels/ 
frequencies,  entering  updated  navigation  data,  and  the  deployment  of  chaff 
and  flares.  Many  other  applications  of  speech  input  also  were  rated  as 
useful.  In  general,  pilot  comments  indicated  a  positive  attitude  about 
speech  input  allowing  them  to  accomplish  certain  actions  in  the  cockpit 
without  having  to  look  inside  and  take  a  hand  off  the  throttle  or  stick. 
Some  pilot  comments  indicated  a  healthy  skepticism  about  implementation 
problems,  such  as:  (1)  maintaining  recognition  accuracy  despite  changes  in 
pilot's  voice,  i.e.,  yelling,  “pulling  Gs,“  etc.,  (2)  preventing  spurious 
recognitions,  i.e.,  “cheer  up"  recognized  as  "gear  up,”  and  (3)  overloading 
the  pilot  with  too  many  communication  responsibi 1 i ties.  A  complete  tran¬ 
scription  of  written  comnents  by  the  pilots  is  given  in  Appendix  D. 

Speech  output  (computer  speech  generation)  was  rated  highly  in  the 
general  area  of  threat  management.  Other  potential  applications  also  were 
rated  as  useful.  However,  a  frequent  comment  by  the  pilots  was  that  speech 
output  should  be  used  sparingly.  It  should  "speak"  only  when  a  critical 
situation  is  detected,  or  upon  request  by  the  pilot.  Several  pilots  sug¬ 
gested  that  an  on/off  switch  would  be  appreciated,  so  they  could  shut-off 
the  system  if  it  became  annoying.  A  suggested  scheme  for  prioritizing 
radio  and  speech-output  communications  is  presented  in  Appendix  8  of  this 
report. 

Implementation  of  speech  1/0  in  future  fighter  aircraft  will  require 
careful  development,  not  only  to  overcome  environmental  factors  such  as 
noise,  vibration,  temperature,  and  acceleration,  but  pilot  workload  issues 
become  paramount  in  designing  an  effective  speech  1/0  system. 

A  number  of  good  candidates  for  speech  1/0  tasks  have  been  identified 
in  the  present  report.  These  should  be  tested  in  a  simulation  enviroment. 
where  system  changes  can  be  effected  quickly  and  easily.  The  simulation 
environment  should  include  realistic  scenarios  of  high  task-loading  to 
determine  the  efficacy  of  confronting  the  pilot  with  combinations  of  speech 
1/0,  radio  communication,  and  vi sual -manual  control.  Guidelines  for  simu¬ 
lation  in  support  of  testing  speech  1/0  are  given  in  Appendix  B. 

It  is  likely  that  the  full  benefit  of  speech  1/0  in  the  cockpit  will 
be  attained  when  it  becomes  part  of  an  "intelligent  systems"  concept  (e.g., 
see  Hopson,  Zachary,  and  Lane,  1981).  This  concept  would  not  only  further 


speech  recognition  accuracy  through  the  processing  of  situational  varia¬ 
bles,  it  would  facilitate  the  dialogue  (be  it  verbal  or  manual)  between  the 
pilot  and  his  cockpit  in  light  of  the  current  mission  events. 

Even  without  the  "intelligent  systems"  concept,  however,  judicious  use 
of  speech  I/O  in  the  fighter  cockpit  will  provide  an  alternate  interface 
which  is  beneficial  to  the  pilot  in  fulfilling  his  mission.  The  present 
data  from  the  F- 16  pilots  support  that  view.  As  stated  at  the  outset,  this 
study,  including  the  conclusions,  are  based  on  the  assumption  that  future 
speech  recognition  technology  will  be  capable  of  performing  in  hostile 
environments  with  very  high  recognition  accuracy. 
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COCKPIT  SPEECH  I/O  RESEARCH  ISSUES* 


During  the  conduct  of  this  study,  a  number  of  issues  were  raised 
either  by  the  F- 16  pilot  participants  or  by  the  authors  regarding  the 
development  and  implementation  of  speech  I/O  in  the  fighter  cockpit. 
Often,  the  simple  questions  are  the  most  important,  both  for  ultimate 
system  performance  and  for  achieving  acceptance  of  the  technology  by  the 
operational  community. 

An  initial  list  of  R&D  questions  was  compiled  after  intensive  inter¬ 
views  with  13  pilots  from  Fighter  Weapons  School  and  422  Operational  Test  & 
Evaluation  Squadron  at  Nellis  AFB.  These  questions  and  issues  are  listed 
on  the  following  pages. 

Subsequently,  the  authors  developed  brief  "issue  definitions"  of 
selected  items  from  that  list.  The  selection  of  items  from  the  list  was 
essentially  arbitrary.  Several  important  and  interesting  questions  were 
never  expanded  to  an  "issue  definition"  due  to  constraints  on  project 
resources.  We  believe  that  the  successful  solution  of  these  issues  is 
necessary  before  the  potential  success  of  cockpit  speech  I/O  can  be 
achieved.  Taken  singly,  some  issues  appear  to  be  strai ghtforward,  but 
their  relationship  and  interaction  may  be  critical  to  the  successful  appli¬ 
cation  of  speech  I/O  in  future  cockpit  designs. 

To  coordinate  a  successful  approach  to  the  solution  of  these  issues  of 
speech  I/O  application,  a  master  plan  must  be  produced  and  controlled  by  a 
central  agency.  This  plan  should  prioritize  the  attack  on  the  R&D  topics 
discussed  herein  and  other  speech  I/O  issues  deemed  applicable.  In  addi¬ 
tion,  the  plan  should  identify  milestones  for  the  application  of  speech 
over  an  extended  timeframe,  i.e.,  the  Far  Term  aircraft.  It  is  important 
that  the  basic  research  components  of  the  plan  do  not  lag  behind  the  more 
applied  R&D  components. 

The  criteria  for  evaluating  the  contribution  of  R&D  issues  should  be 
based  on  its  contribution  to  mission  effectiveness  and  speech  system  per¬ 
formance. 


‘Portions  of  this  discussion  were  included  in  a  presentation/handout  at 
NAECON  '83,  Dayton,  OH,  May  1983. 
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PLANNING 


1.  Speech  I/O  Introduction  Master  Plan 
ROLE  OF  SPEECH  I/O  IN  THE  COCKPIT 


2.  Young  pilot  considerations  on  speech  I/O  implementation 

3.  The  role  of  the  "8-3"  concept,  in  retrofit,  near  term  and  far  term 
aircraft  cockpit  design. 

4.  Application  of  speech  I/O  in  the  design  of  sophisticated  systems 
operating  in  a  down  graded  mode. 

5.  Application  of  speech  I/O  for  single  seat  aircraft  training. 

6.  Designation  of  radar  indicated  targets  by  speech  input. 

7.  Impact  of  speech  I/O  on  proficiency  training. 

8.  Pilot  selected  conditional  settings  of  speech  I/O  commensurate  with 
mission  purpose. 

9.  The  construction  of  interrogative  phrases  which  should  be  used  for  the 
speech  output  system  (commands,  suggestions  or  diagnostic  comments). 

COGNITIVE  LIMITATIONS  AND  INTERPRETATIONS 


10.  Key  word  remembering  performance  for  pilots  in  task  saturated  situa¬ 
tions. 

11.  Use  of  speech  output  to  break  fixation  on  primary  tasks. 

12.  Extent  of  information  flow  by  speech  I/O  before  confusion  sets  in  (How 
much  is  too  much?) 

ACOUSTIC  ENVIRONMENT 

13.  Method  of  reducing  normal  communication  clutter  when  speech  I/O  is 
i ntroduced. 

14.  Use  of  acoustic  patterns  versus  speech  output  message  to  gain  pilot's 
attention. 

OPERATIONAL  ENVIRONMENT 

15.  Use  of  speech  I/O  in  high  G  maneuvers. 

16.  The  value  of  speech  I/O  in  the  single  seat  cockpit  for  high  and  lav 
task/threat  environment. 


YOUNG  PILOTS  AND  SPEECH  I/O  IMPLEMENTATION 


PROBLEM  DEFINITION 

Cockpit  applications  of  speech  I/O,  particularly  in  the  far-term 
aircraft,  must  be  accomplished  with  careful  consideration  of  the  charac¬ 
teristics  of  the  pilot  population.  Assuming  that  the  far-term  aircraft 
becomes  operational  in  10  to  15  years,  the  typical  pilot  who  will  fly  the 
plane  is  now  12  years  old. 

New  technology  is  having  a  great  impact  on  society.  Many  of  today's 
twelve-year  olds  are  already  sophisticated  users  of  personal  computers  and 
video  games.  Manipulation  of  computer  functions  by  voice  is  likely  to  be 
much  less  of  a  "gee  whiz"  capability  to  these  young  pilots  who  are  non¬ 
plussed  by  high  technology  in  the  cockpit.  The  fighter  pilot  of  the  future 
may  be  more  like  Luke  Skywalker  than  John  Wayne. 

Basic  capabilities  of  human  information  processing  will  remain  the 
same;  evolution  is  a  very  slow  process.  Therefore  the  need  for  good  human 
factors  engineering  in  cockpit  design  will  continue.  What  will  change  is 
the  attitude  of  the  typical  fighter  pilot  with  respect  to  rapidly  changing 
technology.  Most  current  pilots  are  justifiably  reluctant  to  endorse  the 
introduction  of  new  technology,  such  as  speech  I/O,  in  the  cockpit.  This 
“Missouri"  attitude  ("show  me  that  it  works")  is  quite  appropriate  to 
prevent  the  proliferation  of  marginally  useful  functions  in  the  cockpit, 
which  are  often  accompanied  by  increased  workload  and  training  time  for  the 
pilot.  However,  once  a  future  cockpit  design  is  solidified  (hopefully 
based  on  human  factors  engineering  analyses)  the  "show  me"  attitude  may  not 
facilitate  the  pilot's  task  of  learning  how  to  use  the  new  systems  adeptly. 

The  next  generation  fighter  pilot  may  be  more  comfortable  with  various 
facets  of  high  technology,  including  speech  I/O,  and  may  have  less  diffi¬ 
culty  learning  to  operate  advanced  systems  to  accomplish  his  mission.  An 
analysis  of  the  expected  characteristics  of  the  next  generation  of  pilots 
is  needed.  It  should  focus  on  the  implications  for  cockpit  design,  systems 
management,  and  pilot  training. 

RESEARCH  CONDITIONS  AND  ENVIRONMENT 

The  projection  of  characteri sties  of  the  future  generation  of  fighter 
pilots  would  best  be  addressed  by  an  organization  familiar  with  the  follow¬ 
ing:  (1)  present  procedures  for  pilot  selection  and  training,  (2)  present 
workload  and  systems  operation  in  the  fighter  cockpit,  (3)  sociological  and 
educational  impact  of  high  technology  on  American  youth.  Familiarity  with 
all  of  these  areas  may  not  exist  currently  within  a  single  organization. 
Forming  a  multidiscipl inary  team  to  address  this  research  issue  is  recom¬ 
mended. 

Potential  spin-offs  from  this  study  would  occur  in  the  areas  of  pilot 
selection  criteria,  and  training  techniques  using  new  technology. 
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TIMING 


The  product  of  this  research  is  needed  when  design  decisions  are  to  be 
made  on  the  use  of  speech  I/O  (and  other  new  technologies)  in  the  cockpit 
of  the  far-term  aircraft.  The  study  would  have  to  be  initiated  at  least 
one  year  prior. 

IMPACT  ON  OTHER  RESEARCH  ISSUES  HEREIN 

The  possible  capability  of  the  next  generation  fighter  pilot  (  a 
"video  arcade  master?")  to  rapidly  learn  to  manipulate  computer-based 
functions  in  the  cockpit  has  widespread  implications  that  cut  across  nearly 
all  of  the  other  research  issues  discussed  herein,  and  are  applicable  to 
many  cockpit  design  problems  besides  speech  I/O. 


79 


i 


THE  ROLE  OF  'INTELLIGENT'  SPEECH  I/O  IN  THE  COCKPIT 


PROBLEM  DEFINITION 

The  concept  of  an  intelligent  speech  I/O  system  is  based  on  the  con¬ 
junction  of  speech  input,  artificial  intelligence  (AI)  and  speech  output. 
The  term  "Backseater  in  a  black  box",  or  B-3,  was  used  in  the  present  study 
to  refer  to  this  concept  of  an  intelligent  system  to  provide  the  pilot  witn 
information  or  to  accomplish  functions  as  requested.  This  type  of  pilot/ 
system  interface  assumes  the  development  of  an  expert  system  that  will 
represent  the  essential  knowledge  of  the  pilot  and  the  hypothetical  back¬ 
seater.  It  would  be  capable  of  "understanding"  the  mission,  tracking  event 
scenarios,  and  "knowing"  aircraft  systems  status. 

The  characteri sties  of  an  intelligent  interface  for  the  cockpit  may  be 
considerably  different  for  the  near-term  and  far-term  aircraft.  In  both 
cases,  the  purpose  of  the  system  would  be  to  provide  services  to  the  pilot, 
to  reduce  his  task  saturation  and  increase  his  effectiveness.  The  intelli¬ 
gent  system  design,  however,  would  be  bound  by  the  characteristics  of  the 
aircraft  systems  and  the  cockpit  design. 

The  research  issue  is  to  develop  the  functional  design  for  an  intelli¬ 
gent  interface  for  the  near-term  and  the  far-term  aircraft. 

An  intelligent  interface  for  the  fighter  cockpit  must  be  based  on 
knowledge  of  the  current  and  future  systems,  pilot  workload,  and  pilot/ 
system  information  exchanges.  The  relationship  between  speech  I/O  and  the 
intelligent  interface  is  an  important  part  of  the  design  process.  This 
research  goes  beyond  the  present  study  by  extending  the  speech  I/O  syntax 
into  the  area  of  AI  and  expert  systems. 

RESEARCH  CONDITIONS  AND  ENVIRONMENT 

This  research  will  require  an  interdisciplinary  team  including  exper¬ 
tise  in  future  aircraft  systems,  speech  I/O,  expert  systems  (knowledge 
representati on) ,  and  human  factors  engineering. 

TIMING 

The  application  of  intelligent  systems  to  cockpit  design  should  begin 
imnediately.  The  proliferation  of  technology  in  the  cockpit  will  soon 
reach  the  stage  where  the  pilot  is  a  definite  limiting  factor  in  system 
operation.  Clever  design  of  the  pilot/system  interface  will  be  an  increas¬ 
ingly  important  requirement.  The  relationship  between  speech  I/O  and  an 
intelligent  interface  is  complex.  Now  is  the  time  to  begin  defining  the 
functional  design  characteristics. 

IMPACT  ON  OTHER  RESEARCH  ISSUES  HEREIN 

Assuming  that  the  "intelligence"  of  a  speech  I/O  interface  will  evolve 
over  time,  the  level  of  the  intelligence  and  the  specific  capabilities  of 
the  system  will  affect  the  entire  role  of  speech  in  the  cockpit. 
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APPLICATION  OF  SPEECH  I/O  FOR  PILOT  TRAINING  IN  SINGLE  SEAT  AIRCRAFT 


PROBLEM  DEFINITION 

In  a  single  seat  fighter,  the  first  flight  is  a  solo.  The  potential 
difficulties  inherent  in  this  fact  have  led  to  progress  in  simulation, 
procedures  trainers,  and  the  costly  development  of  two-seater  trainer 
aircraft. 

Speech  I/O  provides  the  potential  to  aid  the  new  or  transitioning 
pilot  by  allowing  access  to  information  about  procedures,  systems  operating 
parameters,  etc.  The  pilot  could  call  for  these  information  aids  as  needed 
in  flight.  This  concept  is  a  variation  on  the  B-3  systems  discussed  previ¬ 
ously.  Rather  than  a  tactical  team  member  represented  in  the  intelligent 
software,  the  training  version  would  represent  some  of  the  characteristics 
of  an  instructor. 

The  research  issue  is  to  develop  the  functional  design  of  an  intelli¬ 
gent  speech  I/O  system  that  could  aid  and  support  a  pilot  who  is  new  to  the 
ai rcraft. 

Such  a  system  could  provide  a  wide  range  of  services,  from  merely 
providing  data  when  asked,  to  evaluating  pilot  performance  and  suggesting 
techniques  for  improvement.  In  short,  the  entire  range  of  functions  per¬ 
formed  by  an  i nstructor-pi lot  are  candidates  for  including  in  an  on-board 
automated  training  aid. 

Some  of  the  simpler  functions,  such  as  providing  verbal  or  displayed 
information  when  requested,  are  within  the  current  state  of  technology. 
More  complex  functions  requiring  the  development  of  modelling  instructor- 
pilot  functions,  would  be  more  difficult,  but  are  still  within  the  realm  of 
current  technology. 

This  "instructor  in  a  box"  concept  has  the  potential  to  increase 
safety  while  providing  instruction.  Manning  requirements  for  instructor- 
pilots  could  be  reduced,  with  the  associated  cost  savings.  This  type  of 
system  is  not  seen  as  a  replacement,  but  a  supplement,  to  simulation  and 
two-place  aircraft. 

RESEARCH  CONDITIONS  AND  ENVIRONMENT 

Expertise  in  speech  I/O  applications,  pilot  training,  and  instructor¬ 
modelling  will  be  required  to  perform  the  analyses  suggested  by  this  re¬ 
search  issue.  Again,  this  broad  range  of  expertise  may  necessitate  an 
i nterdi sci pi i nary  team.  Researchers  and  scientists  in  speech  1/0  and  AI 
must  be  strongly  grounded  in  the  tactical  environment  to  provide  meaningful 
input  to  this  program.  This  can  be  achieved  by  careful  selection  of  the 
participants  and/or  by  requiring  regular  interaction  with  operational 
personnel . 


31 


i 


TIMING 


Application  of  speech  I/O  as  an  instructional  aid  in  the  single-seat 
fighter  could  be  pursued  in  conjunction  with  the  first  introduction  of 
speech  I/O  in  the  near-term  fighter  cockpit.  An  independent  program  to 
develop  the  sophistication  of  the  system  could  proceed  in  parallel  with  the 
development  of  the  far-term  cockpit. 

IMPACT  ON  OTHER  RESEARCH  ISSUES 

Unknown. 
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THE  IMPACT  OF  SPEECH  I/O  ON  TRAINING  REQUIREMENTS 


PROBLEM  DEFINITION 

The  addition  of  speech  I/O  to  the  cockpit  will  place  the  additional 
burden  on  the  pilot  to  learn  how  to  use  it.  Even  if  speech  I/O  proves  to 
be  very  beneficial  as  a  pilot/system  interface,  pilots  must  learn  how  to 
operate  the  speech  system,  and  how  it  relates  to  traditional  (manual/ 
visual)  modes  of  information  exchange. 

Training  personnel  in  the  use  of  speech  systems  has  been  found  to  be 
critical  for  their  effective  use  (see  the  Canyon  evaluations  of  the  PARTS 
and  ACE  systems,  for  examples).  The  development  of  an  effective  training 
program  will  be  essential  for  the  application  of  speech  I/O  in  the  tactical 
aircraft.  Without  a  good  training  program,  speech  recognition  accuracy 
will  suffer,  system  effectiveness  will  be  reduced,  and  user  acceptance  may 
be  compromised. 

The  research  issue  here  is  to  develop  guidelines  for  pilot  training  in 
the  use  of  speech  I/O  as  the  system  is  under  development.  Even  at  this 
early  stage,  certain  training  principles  in  the  use  of  speech  technology 
can  be  extracted  and  accumulated  for  eventual  inegration  into  a  training 
program.  For  example,  the  requirement  for  speech  sampling  in  a  speaker- 
dependent  recognition  system  carries  with  it  certain  implications  for  pilot 
(user)  training  as  well  as  for  system  design. 

Speech  recognition,  in  particular,  is  sensitive  to  the  variabi 1 ities 
of  the  user.  Recognition  accuracy  commonly  increases  with  practice.  In 
the  same  way  that  a  pilot  must  learn  to  sense  and  feel  the  various  manual 
controls  in  a  new  aircraft,  he  will  have  to  learn  the  limits  of  voice 
variability  that  are  acceptable  to  a  speech  recognition  system.  This 
process  must  be  anticipated  and  encouraged  by  a  good  training  program. 

RESEARCH  CONDITIONS  AND  ENVIRONMENT 

Developing  the  training  issues  for  the  introduction  of  speech  I/O  in 
the  cockpit  should  be  done  by  an  organi zation(s)  with  expertise  in  pilot 
training  and  speech  I/O  applications.  Additionally,  organizations  involved 
in  the  development  of  speech  I/O  in  the  cockpit  should  be  required,  as  an 
ancillary  task,  to  generate  a  list  of  items  or  issues  pertaining  to  pilot 
trai ni ng. 

TIMING 

It  would  be  a  mistake  to  begin  considering  training  issues  only  after 
the  first  aircraft  has  been  delivered  with  speech  technology  in  the  cock¬ 
pit.  They  should  be  considered  at  the  same  time  the  system  is  being  devel¬ 
oped.  A  core  training  program  should  be  developedearly ,  and  a  list  of 
training  issues  should  be  added  as  the  system  evolves  during  RDT&E. 
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IMPACT  ON  OTHER  RESEARCH  ISSUES 

All  of  the  research  and  development  programs  discussed  herein  will 
include  factors  that  affect  the  way  the  system  is  to  be  designed  or  oper¬ 
ated.  These  factors  become  candidates  for  items  in  a  training  program  for 
the  pilot,  as  a  user  of  the  system. 
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PILOTS  MEMORY  FOR  WORDS  DURING  HEAVY  WORKLOAD 


PROBLEM  DEFINITION 

A  pilot's  memory  for  the  location  of  buttons  or  switches  is  essential 
for  operating  aircraft  effectively.  Similarly,  his  memory  for  key  words 
and  commands  will  be  essential  for  proper  operation  of  a  speech  input 
system.  The  cognitive  performance  of  pilots  (memory,  response  time,  error 
probability,  etc.)  will  be  different  for  vi sual /manual /spat i al  tasks  than 
for  auditory/verbal /serial  tasks.  How  these  two  sets  of  skills  are  affect¬ 
ed  by  task  loading  and  combat  stress  is  important  for  the  design  of  the 

pilot/system  interface. 

If  remembering  vocabulary  items  or  word  sequence  constraints  is  a 

problem  in  a  high  threat  environment,  for  example,  the  interface  design 
must  account  for  it.  Examples  of  design  options  are  whether  to  have  a 
manual  or  speech  input  for  certain  functions  (or  both),  allowing  alterna¬ 
tive  words  to  be  used  to  accomplish  the  same  function,  and  developing  the 
speech  syntax  with  flexibility  to  avoid  strict  rules  of  word  sequences. 

The  research  issue  is  to  analyze  and  define  the  cognitive  and  memory 
processes  of  the  fighter  pilot,  and  how  they  are  influenced  by  stress, 

these  analyses  should  be  done  with  the  goal  of  optimizing  the  cockpit 

design,  and  particularly  the  integration  of  speech  I/O  in  the  cockpit.  The 
implications  for  vocabulary  size  and  syntax  development  are  directly  appli¬ 
cable.  Information  relevant  to  learning/training  also  should  result  from 
these  analyses. 

If  the  cognitive  load  created  by  remembering  and  using  the  speech 
input  vocabulary  is  so  high  that  it  reduces  the  pilot's  performance  in 
other  areas  (i.e.,  target  detection,  communications  i nterpretat i on ,  etc.), 
this  would  have  great  importance  for  the  application  of  speech  input.  The 
same  applies  for  the  cognitive  load  created  by  perceiving  and  understandi ng 
speech  output  messages  in  a  busy  situation. 

RESEARCH  CONDITIONS  AND  ENVIRONMENT 

This  work  should  be  led  by  an  entity  with  expertise  in  cognitive 
science  (a  subcategory  of  Experimental  Psychology)  in  close  cooperation 
with  people  who  are  familiar  with  the  fighter  pilot's  task,  and  with  the 
speech  technology.  This  could  be  done  by  some  combination  of  university 
scientists  and  operational  personnel,  or  by  an  organization  with  a  human 
factors  orientation,  bridging  the  three  necessary  areas  of  expertise. 

TIMING 

The  knowledge  of  expected  effects  of  stress  verbal  memory  and  the 
operation  of  a  speech  I/O  system  should  be  obtained  well  before  this  tech¬ 
nology  is  implemented  in  operational  aircraft. 
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IMPACT  ON  OTHER  RESEARCH  ISSUES 


Memory  limitations  and  cognitive  capacity  in  stressful  situations  may 
interact  with  several  of  the  research  issues  discussed  herein,  e.g.,  young 
pilots,  the  intelligent  interface,  training  objectives,  and  the  high  threat 
environment . 
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APPLICATION  OF  SPEECH  I/O  IN  THE  DESIGN  OF 
SOPHISTICATED  SYSTEMS  OPERATING  IN  A  DOWNGRADED  MODE 


PROBLEM  DEFINITION 

Increasingly  sophisticated  automated  systems  will  be  provided  in 
future  aircraft  design.  Such  systems,  while  substanti al ly  redundant  by 
design,  will  continue  to  require  the  intervention  by  the  pilot  when  the 
system  is  operating  in  a  downgraded  mode.  The  frequency  of  downgrading 
will  be  low  due  to  high  reliability.  Accordingly,  the  pilot  exposure  to 
downgraded  system  operation  will  be  low. 

It  has  been  proposed  that  the  internal  fault  monitoring  system  be 
interfaced  with  the  speech  I/O  to  provide  communications  between  the  system 
and  the  pilot  (I)  to  minimize  the  disruptiveness  of  the  malfunction,  (2)  to 
request  the  pilot  to  fake  specific  actions  ensuring  mission  safety  and  (3) 
to  lead  the  pilot  through  a  dowgraded  system  checklist  procedures  (if 
required).  Speecn  I/O  is  a  more  flexible  interface  witn  the  pilot  than 
using  extensive  verbage  on  a  MFD  for  the  same  purpose. 

RESEARCH  CONDITIONS  AND  ENVIRONMENT 

This  research  should  be  carried  out  in  large  systems  houses  with 
experience  of  'integrated  aircraft  systems.  The  work  should  be  carried  out 
under  the  direction  of  an  applied  cockpit  speech  I/O  research  group. 

TIMING 

This  research  should  be  pursued  for  far  term  aircraft  only  (1990- 
2000).  However,  the  principles  for  the  speech  I/O  interface  with  sophis¬ 
ticated  systems  need  to  be  developed  now,  so  that,  a  design  guide  is  avail¬ 
able  to  future  system  designers. 

IMPACT  ON  OTHER  RESEARCH  ISSUES  HEREIN 

The  main  impact  will  be  on  the  vocabulary  and  syntax  development  so 
that  the  pilot  will  interface  easily  with  the  degraded  system.  A  secondary 
effect  will  be  the  extent  of  initial  and  recurrent  training  the  pilot  will 
require  to  use  a  speech  I/O  interface  infrequently. 


PILOT  SELECTION  OF  SPEECH  I/O  FUNCTIONS 
COMMENSURATE  WITH  MISSION  PURPOSE 


PROBLEM  DEFINITION 

Some  pilots  may  desire  to  use  speech  I/O  more  frequently  than  others. 
For  instance,  if  the  mission  involves  serveral  friendly  aircraft  with 
expected  communication  clutter,  the  pilot  may  desire  to  use  the  speech  I/O 
only  for  new  threat  information  and/or  for  verbal  checklists  when  an  emer¬ 
gency  situation  exists.  If  the  next  mission  involves  a  single  long,  low 
level,  night  reconnaissance  into  enemy  territory  with  little  communica¬ 
tions,  then  he  may  elect  to  have  the  total  speech  I/O  working  for  him.  In 
the  far  term  concept  the  speech  I/O  interface  could  encompass  all  discrete 
manual  actions  which  the  pilot  may  make  in  a  cockpit. 

RESEARCH  CONDITIONS  AND  ENVIRONMENT 

In  the  near  term,  this  issue  of  pilot  selection  of  functions  may  be 
resolved  using  a  tactical  pilot  population  in  a  full  mission  training 
simulator  in  which  speech  I/O  has  been  provided  for  those  items  involving 
high  threat,  high  workload,  eyes  out  environments.  The  analysis  of  pre¬ 
mission  selection  of  speech  I/O  functions  could  determine  the  need  for 
individual  pilot  selection.  The  pilot  population  used  for  this  study 
should  encompass  both  the  unexperienced  and  very  experienced  pilots. 

For  the  far  term,  speech  I/O  interface  should  be  considered  as  an 
integral  part  of  the  design  of  future  tactical  aircraft  cocpits.  Speech 
I/O  selection  then  becomes  a  fallout  of  individualized  design  (assuming 
that  customization  is  a  viable  alternative). 

TIMING 

Resolution  of  this  issue  in  the  near  term  is  important  because  it  has 
a  profound  affect  on  the  speech  I/O  interface  implementation  concept  for 
aircraft  in  the  1985-1990  time  frame. 

For  the  far  term,  design  guidelines  for  speech  I/O  interface  imple¬ 
mentation  should  be  available  by  the  end  of  1985  to  provide  guidance  on  p- 
ilot/cockpit  function  selection,  both  pre-flight  and  in-flight. 

IMPACT  ON  THE  RESEARCH  ISSUES  HEREIN 

The  impact  of  this  basic  issue  is  pervasive  in  respect  to  speech  I/O 
interface  implementation.  It  has  its  origin  in  the  designer’s  ability  to 
demand  pilot  functional  selections  in  an  increasingly  automated  environ¬ 
ment.  To  date  no  rigorous  man/machine  allocation  procedure  has  been  de¬ 
veloped  to  make  such  selection  in  a  consistently  meaningful  manner. 
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METHOD  OF  REDUCING  NORMAL  COMMUNICATIONS  CLUTTER 
WHEN  SPEECH  I/O  INTERFACE  IS  INTRODUCED 


PROBLEM  DEFINITION 

Radio  telephone  (R/T)  communication  clutter  is  normal  to  tactical 
aircraft  operations  when  more  than,  say,  three  aircraft  are  involved  in  a 
mission.  Pilots  do  their  own  filtering  and  reject  R/T  conmunicated  infor¬ 
mation  which  is  not  useful  to  them.  Pilots  have  expressed  concern  that 
speech  output  from  the  aircraft's  system  could  cause  distraction  in  a 
communication  clutter  situation  and  under  some  circumstances  would  inter¬ 
fere  with  the  mission. 

The  usual  technique  to  reduce  communications  clutter  is  to  provide 
more  dedicated  communications  channels  to  the  user  population.  The  expan¬ 
sion  of  this  scheme  to  help  find  a  place  for  the  speech  I/O  interface  in 
the  cockpit  information  flow,  has  obvious  limitations. 

Knowing  that  the  need  for  tactical  communications  will  always  exist 
and  that  speech  output  also  needs  to  have  access  to  the  pilot's  brain,  the 
issue  becomes  to  develop  different  strategies  of  aural  communi cat ions.  For 
example,  can  the  R/T  communications  clutter  be  reduced  to  only  those  air¬ 
craft  within  a  certain  limited  air  space  and  range  volume?  Alternatively, 
can  air-to-air,  air-to-ground  and  ground-to-air  communications  be  allocated 
to  a  discrete  address  systems  as  intended  for  use  in  future  air  traffic 
control  systems. 

RESEARCH  CONDITIONS  AND  ENVIRONMENT 

Psychological  research  is  required  on  how  pilots  derive  information 
from  aural  communication.  A  question  like,  "can  the  right  ear  be  used  for 
R/T  and  the  left  ear  used  for  speech  sytem  outputs?"  needs  to  be  answered 
in  a  laboratory  environment.  Improvement  of  R/T  communications  technqiues 
involving  space  limiting  or  discrete  addressing,  requires  the  expertise  of 
communication  engineers. 

TIMING 

The  timing  to  find  a  solution  is  "now."  It  is  the  authors'  view  that 
the  continuing  problem  of  communications  clutter  in  the  real  world  tactical 
aircraft  environment  may  limit  the  usefulness  of  cockpit  speech  output. 

IMPACT  ON  OTHER  RESEARCH  ISSUES  HEREIN 

This  issue  is  directly  related  to  "Prioritization  of  Speech  Output 
Messages"  and  to  cognitive  workload  issues. 
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THE  VALUE  OF  SPEECH  I/O  IN  THE  SINGLE  SEAT  COCKPIT 
FOR  HIGH  ANO  LOW  TASK/THREAT  ENVIRONMENT 


PROBLEM  DEFINITION 

Some  tactical  aircraft  pilots  consider  that  if  speech  I/O  is  to  be 
introduced  in  the  future  cockpit  designs,  it  should  be  limited  to  those 
functions  which  are  frequently  used  in  a  high  workload,  high  threat,  hands- 
on-throttle-and-stick  heads  out  environment.  Other  tactical  pilots  see  the 
speech  I/O  interface  having  a  role  which  supports  the  general  piloting  task 
independent  of  mission  phase  (analogous  to  a  second  pilot  in  the  back 
seat). 

The  design  of  speech  I/O  interface  in  the  tactical  cockpit  lacks  a 
rigorous  man/machine  allocation  procedure.  This  same  problem  exists  in  the 
design  of  all  aircraft  automated  systems  which  interface  with  the  pilot. 
The  issue  then  is  to  develop  a  rigorous  allocation  program  which  will 
perform  for  every  aircraft  system  configuration  and  mission  phase.  If  this 
issue  is  insol uable,  then  speech  I/O  should  be  allowed  to  propogate  through 
all  cockpit  functions  as  an  alternate  mode  with  the  choice  of  use  being 
left  to  pilot  choice  dependent  upon  prevailing  needs  and  operational  condi¬ 
tions. 

RESEARCH  CONDITIONS  AND  ENVIRONMENT 

For  the  overall  and  far  term  solution,  if  an  appropriate  man/machine 
task  allocation  program  for  future  aircraft  cockpit  design  exists  then  the 
speech  I/O  interface  utilization  problem  should  be  emphasized  in  the  con¬ 
text  of  that  research. 

In  the  near  term  a  study  similar  to  that  of  "development  of  speech 
input/output  interfaces  for  tactical  aircraft"  should  be  undertaken  by  an 
independent  aviation  human  factors  organization  who  have  demonstrated 
understanding  of  the  tactical  aircraft  environment.  This  study  should  be 
directed  at  reducing  the  wide  differential  of  opinion  which  the  tactical 
aircraft  community  has  about  speech  I/O.  The  pragnatism  of  the  experienced 
pilot  needs  to  be  integrated  with  the  innovativeness  of  youthful  community 
(the  latter  stand  to  be  the  primary  users  of  speech  I/O). 

TIMING 

The  industry  need  for  guidance  on  this  matter  exists  now.  In  the 
longer  term,  the  solution  to  this  problem  is  likely  to  be  a  product  of  the 
speech  I/O  configuration  research  for  the  far  term  aircraft. 

IMPACT  ON  OTHER  RESEARCH  ISSUES  HEREIN 

The  impact  of  the  subject  issue  is  far  reaching  as  the  issue  is  focal 
to  the  extent  the  speech  I/O  interface  propogates  into  the  cockpit's  func¬ 
tional  operation. 
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PRIORITIZATION  rF  SPEECH  OUTPUT  MESSAGES 


PROBLEM  DEFINITION 

Assuming  that  speech  output  messages  are  communicable  to  the  pilot 
through  the  communications  clutter,  the  speech  output  system  can  be  ex¬ 
pected  to  generate  more  than  one  message  at  any  specific  instance.  The 
importance  of  each  message  becomes  the  driver  for  a  speech  output  message 
prioritization  scheme. 

A  generic  approach  is  required  for  such  a  prioritization  scheme  such 
that  it  is  not  type  of  aircraft  or  type  of  system  specific. 

Consideration  needs  to  be  given  to  the  necessity  for  an  incoming  R/T 
signal  override  function  at  the  time  a  speech  output  content  is  being 
communicated  to  the  pilot. 

At  issue  is  the  protocol  for  message  prioritization,  overriding  and 
message  repeating  in  a  cockpit  environment.  The  issue  includes  the  priori¬ 
ty  which  should  be  given  to  the  traditional  buzzers  and  tones  that  will 
continue  to  exist  in  most  cockpits. 

RESEARCH  CONDITIONS  AND  ENVIRONMENT 

A  study  on  potential  cockpit  messages,  generic  style  and  content,  plus 
a  literature  search  into  message  communication  priorities  is  required. 
Experiments  using  suitable  part  task  simulation  would  bring  potential 
prioritization  schemes  into  focus  for  cockpit  applicaiton.  Full  mission 
simulation  using  preferred  schemes  would  then  be  required  for  system  design 
confirmation. 

TIMING 

Message  prioritization  becomes  increasingly  important  as  the  role  of 
the  speech  output  interface  expands  in  the  cockpit.  The  need  for  systema¬ 
tic  work  on  this  issue  exists  right  now. 

IMPACT  ON  OTHER  RESEARCH  ISSUES  HEREIN 

Message  prioritization  affects  the  interaction  of  speech  input  and 
output,  the  design  of  "intelligent"  speech  I/O,  and  the  cognitive  loading 
of  the  pilot. 
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RESEARCH  FEATURES  AND  DESIGN  REQUIREMENTS 
FOR  SPEECH  INTERFACED  TACTICAL  AIRCRAFT  SIMULATION 


Introduction 

This  document  attempts  to  provide  a  framework  for  simulating  a  speech 
input/output  mode  to  be  used  in  future  tactical  aircraft  cockpits.  A  full 
mission  simulator  is  not  required  for  implementation  of  the  speech  I/O 
mode,  but  it  is  not  precluded. 

The  simulation  framework  is  directed  at  demonstrating  and  evaluating  a 
speech  mode  in  an  "eyes  out"  and/or  "high  threat"  environment  which  is  the 
focus  of  the  aircraft  system  functions  selected  for  simulation. 

The  assumptions  used  in  developing  this  document  are  summarized  as 
follows: 

1.  The  simulation  will  be  used  to  demonstrate  the  feasibility  and 
evaluate  the  performance  of  selected  speech  mode  functions  for  use 
in  an  aircraft  similar  to  the  F- 16  in  the  1985-1990  timeframe. 

2.  A  conventional  interface  between  the  pilot  and  aircraft  systems 
will  be  provided  for  the  selected  speech  mode  functions  and  other 
system  functions  required  to  conduct  eyes  out/high  threat  mission 
scenarios. 

3.  The  interface  will  include  two  multi-function  displays,  part  of 
which  will  be  dedicated  for  speech  I/O  message  feedback. 

4.  The  intent  and  capability  of  the  speech  mode  functional  interface 
will  be  selectable  prior  to  the  conduct  of  each  demonstration, 
exercise  or  experiment. 

5.  The  simulation  will  be  suitable  for  the  investigation  of  applied 
speech  I/O  research  issues  identified  in  Appendix  B  of  this 
report.  As  such,  the  simulation  will  include  a  speech  mode  per¬ 
formance  algorithm  and  data  recording  capability. 


Thirteen  generic  types  of  speech/aircraft  system  functions  will  be 
simulated  as  follows: 


INPUT  -  Function  Selection  -  use  of  speech  recognition  to  com¬ 

mand  all  mode/function  switching,  knob  and  lever  selec¬ 
tion  in  the  cockpit.  (See  SIMIN  No.  1,  3,  and  4  below.) 

-  Request  Information  Display  -  use  of  speech  recognition 
to  request  the  display  of  various  data  on  the  multifunc¬ 
tion  displays.  (See  SIMIN  No.  6  below.) 
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-  Data  Entry  -  use  of  speech  recognition  to  enter  data 
into  the  aircraft  systems.  (See  SIMIN  No.  2,  5,  and  7 
below. ) 

OUTPUT  -  System  Advisory  -  use  of  speech  generation  to  convey 
normal  system  events.  (See  SIMOUT  No.  5  below.) 

-  System  Abnormality  -  use  of  speech  generation  to  convey 
abnormal  system  events  which  do  not  impair  inmediate 
safety  of  flight.  (See  SIMOUT  No.  3  and  6  below.) 

-  System  Critical  -  use  of  speech  generation  to  convey 
critical  system  events  which  impair  the  immediate  safety 
of  flight.  (See  SIMOUT  No.  9  below.) 

-  Mission  Advisory  -  use  of  speech  generation  to  convey 
normal  mission  events.  (See  SIMOUT  No.  2  below.) 

-  Mission  Abnormality  -  use  of  speech  generation  to  convey 
mission  events  which  are  not  planned.  (See  SIMOUT  No.  6 
below. ) 

-  Mission  Critical  -  use  of  speech  generation  to  convey 
high-threat  events  which  are  critical  to  the  success  of 
the  mission.  (See  SIMOUT  No.  1,  4  and  7  below.) 

INTERACTION  -  Command  feedback  use  of  speech  generation  response  to  a 
previous  speech  input  command  to  the  aircraft  systems. 
(See  SIMOUT  No.  10  below.) 

-  System  Status  -  use  of  speech  recognition  and  generation 
to  determine  the  status  of  aircraft  systems  including 
automatic  checking  systems.  (See  SIMI/0  No.  2  below.) 

-  Diagnostic  Response  -  use  of  speech  recognition  and 
generation  to  help  diagnose  and  remediate  the  effects  of 
a  downgraded  or  failed  system.  (See  SIMI/0  No.  4 
below. ) 

-  System  Information  Processing  -  use  of  speech  recogni¬ 
tion  and  generation  to  provide  an  interactive  interface 
for  calculation  and  retrieval  of  system  and  mission 
information.  (See  SIMI/0  No.  1  and  3  below.) 

The  aircraft  functions  selected  for  speech  I/O  simulation  which  follow 
are  predicated  on  the  results  of  the  F - 16  pilot  community  survey  described 
in  Section  IV  of  this  report.  The  question  numbers  from  the  pilot  survey 
are  indexed  to  the  suggested  "SIM"  functions  in  Table  B-l. 

SIMIN  1.  Perform  chaff  or  flare  release. 

This  will  control  the  order  of  release  in  accordance  with  a 
preflight  programmed  schedule.  Simulated  release  of  the  chaff 
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TABLE  B-l.  SIMULATION  SOURCE  SELECTION  SUMMARY 


SPEECH  INPUT 

QUESTIONNAIRE  QUESTION  NUMBER 

SIMIN 

1 

24 

SIM  I N 

2 

33,  34,  35 

SIMIN 

3 

25,  31,  32,  37,  38,  39,  47,  48,  49 

SIMIN 

4 

45 

SIMIN 

5 

36 

SIMIN 

6 

27,  28,  29,  30,  44,  46 

SIMIN 

7 

43 

SIMIN 

8 

42 

SPEECH  OUTPUT 

SIMOUT  1 

2,  3,  4,  19 

SIMOUT 

2 

6 

SIMOUT 

3 

17 

SIMOUT 

4 

5 

SIMOUT 

5 

1 

SIMOUT 

6 

7,  9,  15,  20,  8 

SIMOUT 

7 

23 

SIMOUT 

8 

10 

SIMOUT 

9 

21 

SIMOUT 

10 

11 

SPEECH  INPUT/OUTPUT 

SI  MI /O  1 

50,  52 

SIMI/O 

2 

40,  41 

SIMI/O 

3 

53 

SIMI/O 

4 

16 
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or  flares  may  be  indicated  on  the  applicable  MFD  as  an  alterna¬ 
tive  to  speech  recognition  feedback.  This  alternative  will  be 
selected  by  a  keyboard  entry  into  the  simulation  host  computer. 

SIMIN  2.  Initiate  channel  or  frequency  changes  on  nominated  radio  equip¬ 
ment  (VHF,  UHF,  TACAN,  etc.). 

Actual  channel  or  frequency,  as  provided  on  an  indicator  de¬ 
signed  for  this  purpose,  may  be  used  as  an  alternative  to 
generated  speech  feedback.  This  alternative  will  be  selected 
by  a  keyboard  entry  into  the  simulation  host  computer. 

SIMIN  3.  Command  the  modes  of  mission  critical  equipment  -  like  “ECM 
ON , “  "MASTER  ARM  ON,"  "LANTIRN  SEARCH,"  "MISSILE  UNCAGE," 
"AIR-TO-AIR,"  etc."  and  "MFD  FORMATS." 

The  repetoire  of  mission  critical  modes  selection  will  be 
dependent  on  the  extent  and  sophistication  of  the  simulation 
provided  for  this  purpose.  However,  each  mode  should  be  selec¬ 
table  through  the  simulator  host  computer  data  entry  keyboard. 

SIMIN  4.  Control  interior  and  exterior  lighting  levels. 

Light  levels  of  annunciator  switches,  indicator  lights  etc., 
will  be  provided  automatically  as  feedback.  Automatic  movement 
of  incremental  controls  like  dimmers  need  not  be  provided. 

SIMIN  5.  Input  pre-mission  data  such  as  navigation  point  coordinates, 
fuel  load,  weapons  types  and  stores  configuration. 

The  same  data  may  be  altered  in  flight  by  identifying  the 
variable(s)  to  be  changed.  Confirmation  feedback  for  the 
verbally  entered  data  may  be  by  speech  generation  or  MFD  dis¬ 
play,  or  both. 

Selection  of  the  desired  level  of  feedback  will  be  a  keyboard 
entry  into  the  host  computer. 

SIMIN  6.  Call  up  information  to  be  displayed  on  the  secondary  MFD. 

•  Information  will  include  route,  approach,  JTIDS,  checklists, 
available  MFD  formats,  etc. 

SIMIN  7.  Updating  present  position  over  the  top  of  a  nominated  naviga¬ 
tion  point  by  saying,  for  example,  "DP3  MARK."  Confirmation 
that  the  update  took  place  can  be  displayed  on  the  secondary 
MFD  and/or  be  provided  by  the  speech  generation  system. 

SIMIN  8.  Request  aircraft  system  critical  information  like:  "fuel 
remaining,"  "stores  remaining,"  etc. 

Confirmation  of  speech  recognition  which  may  be  speech  gene¬ 
rated  and/or  displayed  on  the  secondary  MFD. 

Selection  of  the  desired  level  of  feedback  will  be  a  keyboard 
entry  into  the  host  computer. 
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The  selected  aircraft  functions  for  output  simulation  are: 


SIMOUT  1.  Provide  detailed  threat  warning  information. 

This  will  cover  the  initial  acquisition  of  each  threat,  identi¬ 
fication  when  known,  bearing  sector  relative  to  the  aircraft’s 
nose  and  an  update  of  threat  status.  Selection  of  threat 
information,  type,  bearing,  status,  and  their  message  queue 
priorities  will  be  provided  by  a  keyboard  entry  into  the  host 
simulation  computer  prior  to  the  commencement  of  each  exercise. 


SIMOUT  2.  Prompt  the  necessity  to  change  operational  identification. 

This  will  be  a  "universal  time”  oriented  general  message  which, 
for  example,  advises  the  pilot  to  change  IFF  code(s)  to  the 
appropriate  setting  (per  the  flight  plan).  A  keyboard  entry 
into  the  host  computer  prior  to  the  commencement  of  each  exer¬ 
cise  will  set  the  appropriate  time.  Message  queue  priority 
normally  will  be  kept  constant. 


SIMOUT  3.  Enunciate  master  warnings  and/or  caution  occurrences. 

There  will  be  two  levels  of  output.  The  upper  level  will 
include  those  primary  system  warnings  which  are  not  in  the 
pilot's  eye  field  of  view,  such  as  master  caution,  engine  fire, 
etc.  The  lower  level  will  provide  the  detailed  cause  of  why 
the  master  warning  or  caution  occurred,  such  as  an  "avionics" 
caution.  A  keyboard  entry  into  the  simulation  host  computer 
will  determine  which  level  of  output  will  be  used  in  the  exer¬ 
cise  and  it's  message  queue  priority. 

SIMOUT  4.  Inform  when  bingo  fuel  has  been  reached. 

Message  queue  priority  normally  will  be  kept  constant. 

SIMOUT  5.  Provide  specific  altitude  readout  below  specified  altitudes. 


This  will  be  a  multiple  level  output  which  can  be  determined  by 
the  aircraft's  external  control  configuration  and/or  altitude. 
Control  configuration  levels  include  gear  and  flap  position. 
Altitude  levels  are  suggested  as: 

>  10,000  ft.  to  1,000  ft.  -  every  1,000  ft. 

>  1,000  ft.  to  50  ft.  -  every  100  ft. 

>  50  ft.  to  10  ft.  -  every  10  ft. 

A  keyboard  data  entry  into  the  simulation  host  computer  will  be 
used  to  select  the  levels  and  message  queue  priority  to  be  used 
in  each  exercise. 


SIMOUT  6.  Report  degradation  of  mission  critical  systems. 

This  output  is  intended  to  identify  degradation  of  systems 
which  are  critical  to  the  safety  and  success  of  a  mission. 
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SIMOUT 


SIMOUT 


SIMOUT 


Such  systems  are  expected  to  include  a  self  contained  perfor¬ 
mance  monitoring  device  which  will  indicate  when  specific 
criteria  are  exceeded. 

Systems  considered  to  be  in  this  category  include,  threat 
warning,  ECM,  navigation,  flight  control,  fuel  distribution, 
main  power  plant,  etc.  For  example,  a  flight  control  system 
could  be  operating  with  a  failed  servo. 

A  keyboard  entry  into  the  simulator  host  computer  will  be  used 
to  select  which  system  is  selected  and  the  message  queue  pri¬ 
ority  to  be  used. 

7.  Provide  warning  of  laser  designated  target  masking  through 
evasive  maneuvering. 

This  will  inform  the  pilot  when  he  has  masked  a  predesignated 
target  which  is  scheduled  to  be  struck  by  an  automatically 
released  store. 

Message  priority  will  normally  be  kept  constant. 

3.  Prompt  -  the  necessity  to  carry  out  routine  mission  oriented 
checks. 

These  will  be  navigation  position  or  universal  time  oriented 
general  message  which,  for  example,  advises  the  pilot  to  con¬ 
duct  a  fence  check. 

Message  queue  priority  will  normally  be  kept  constant. 

9.  Inform  corrective  action  and  direction  of  application  when  the 
aircraft's  aerodynamic  performance  envelope  is  exceeded. 

This  may  be  impractical  to  simulate  for  demonstration  unless 
adequate  visual  cues  are  available. 

10.  Feedback  verbal  input  information  which  has  been  recognized  by 
the  speech  system  or  is  in  the  process  of  being  recognized. 

There  will  be  three  levels  of  control  for  this  process.  The 
first  will  repeat  the  input  message  when  it  has  been  recog¬ 
nized.  The  second  level  will  include  the  first  level  feature 
but  in  the  event  that  the  verbal  input  is  not  recognized  first 
time,  a  generated  speech  output  will  say  "Say  Again."  A  third 
level  of  control  will  inhibit  any  verbal  feedback  to  the  pilot 
in  favor  of  a  visually  displayed  message  (see  also  the  sections 
on  input  control  and  feedback). 

A  keyboard  entry  into  the  simulator  host  computer  will  be  used 
to  control  the  level  of  feedback  to  be  provided. 


99 


( 


The 

SIMI/O 


SIMI/O 


SIMI/O  3 


SIMI/O  4 


selected  interact i ve  functions  for  simulation  are: 

L.  Request  critical  flight  planning  information  like  "fuel  home," 
"fuel  to  alternate. " 

This  assumes  a  nominal  cruise  and/or  descent  profile  from 
present  position  to  touchdown.  It  will  require  inputs  from  the 
navigation,  fuel,  air  data,  and  stores  management  equations  for 
the  simulation  host  computer  to  make  the  determination. 

Message  priority  in  the  speech  output  queue  will  be  kept  con¬ 
stant. 

!.  Request  status  of  automatic  aircraft  system  checkoff  like 
"fence  check  status." 

This  requires  multiple  system  inputs  to  be  available  to  a 
simple  equation  which  represents  the  checklist  items. 

Actions  still  required  to  be  made  by  the  pilot  will  then  be 
speech  generated  and/or  displayed  on  the  secondary  MFD  until 
the  procedure  is  completed. 

No  action  by  the  pilot  will  stop  further  speech  generated 
requests. 

Message  priority  in  the  speech  output  queue  will  be  kept  con¬ 
stant. 

.  Request  future  flight  plan  information  like  “features  for  DP 
3";  typical  response  would  be  "one  tree  hill." 

This  requires  prior  identification  of  each  navigation  check 
point  in  a  data  file  held  by  the  simulation  host  computer. 
This  file  would  be  a  “scratch"  type  file  for  each  mission  to  be 
conducted  (this  would  be  in  contrast  to  the  general  route  and 
approach  plate  information  file  which  could  be  called  on  by 
route,  approach,  or  runway  ident). 

If  more  than  twenty-four  characters  long,  data  so  requested 
will  be  speech  generated.  If  less  than  twenty- four  characters 
long,  they  may  be  displayed  on  the  secondary  MFD.  In  the 
latter  case  generated  speech  will  also  prompt  the  pilot  to  "see 
MFD." 

•  Respond  to  pilot's  request  for  information  to  help  resolve 
degraded  performance  of  an  aircraft  system. 

This  is  intended  to  suggest  remedial  solutions  which  may  not  be 
readily  recalled  by  the  pilot  in  high  threat  or  high  workload 
situations. 

The  feature  requires  storage  by  the  simulation  host  computer  of 
fault  trees  for  flight  critical  and  mission  ciritical  systems. 
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Speech  generation  will  be  used  if  the  solution  message  is  long. 
The  secondary  MFD  may  be  used  if  the  messges  is  less  than 
twenty-four  characters  in  length.  In  the  latter  case,  generat¬ 
ed  speech  will  prompt  the  pilot  to  "see  MFD." 

Input  Control 

Speech  input  will  be  through  a  boom  headset  microphone  which  is  com¬ 
patible  with  the  speech  recognition  equipment  used  in  the  simulation.  The 
microphone  normally  will  be  inactive.  A  special  microphone  key  will  be 
fitted  to  the  simulated  engine  throttle  control  to  enable  keying  to  the 
“ON"  condition. 


Selected  speech  functions  available  to  the  "pilot"  for  each  exercise 
will  be  preselected  by  the  simulator  technician  using  a  data  entry  keyboard 
associated  with  the  simulation  host  computer. 


Verification  of  each  verbal  input  analyzed  by  the  speech  recognition 


-  at  which  time  a  speech  output  which  says 
“say  again,"  will  be  initiated. 


system 

wi 1 1  be  made  i n  one  of 

1. 

Not  recognized 

2. 

Recognized  but 
requires  further 
systems  intelligence 
to  fulfill  command 

3. 

Recognized 

At  which  time  a  1/2  second  500  HZ  audio 
tone  will  be  initiated  to  indicate  that 
action  is  taking  place. 


-  at  which  time  two  1/2  second  500  HZ  audio 
tones  with  a  1/2  second  separation  will  be 
initiated.  In  addition,  the  recognized 
message  will  be  displayed  unobtrusively  on 
the  secondary  MFD  and  will  remain  displayed 
until  replaced  by  the  next  recognized 
verbal  command. 


Functional  and  situational  discrimination  logic  (often  referred  to  as 
syntax,  speech  understanding,  or  artificial  intel 1 igence)  will  be  provided 
down  line  from  the  speech  recognition  system  by  the  host  computer.  The 
“say  again"  interrogative  can  be  initated  by  either  the  speech  recognizer 
or  the  functional  situational  logic.  The  audio  tones  will  be  initiated  by 
discrete  output(s)  from  the  functional  and  situational  logic. 


Output  Control 

Output  messages  will  be  transmitted  through  the  simulator  audio  chan¬ 
nel  used  for  communications  with  the  pilot.  The  protocol  for  speech  output 
queueing  will  have  four  levels  of  priority  and  radio  transmissions  will 
override  system-generated  messages.  The  four  levels  of  queueing  protocol 
will  be  as  follows: 


Priority  #1  -  Ultra  important  messages  which  assist  in  the  safe  con¬ 
duct  of  the  mission.  These  messages  will  override  any 
other  message  being  transmitted  by  the  speech  generation 
sysem  even  if  the  message  is  incomplete. 
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Priority  42  -  Very  important  messages  which  assist  in  the  management 
of  the  extraordinary  events  experienced  during  a  mis¬ 
sion,  e.g.,  flight  control  system  servo  failure. 

Priority  4$  -  Important  messages  which  assist  in  the  management  of 
ordinary  events  experienced  in  flight,  e.g.,  endurance/ 
fuel  remaining. 

Priority  #4  -  Routine  messages  which  assist  the  pilot  in  routine  work, 
e.g.,  reciting  checklists. 

Each  Priority  42  through  #4  speech  generated  message  will  be  completed 
before  the  next  message  is  commenced,  unless  a  Priority  #1  message  inter¬ 
venes. 

When  more  than  one  message  is  in  the  queue,  the  priority  will  be 
adjusted  before  the  next  message  is  transmitted.  Messages  which  have  been 
transmitted  incompletely  will  be  reassigned  to  their  correct  priority  posi¬ 
tion  and  retransmitted  as  complete  messages.  Messages  which  are  delayed  by 
the  priority  hierarchy  by  more  than  20  seconds  will  not  be  transmitted  and 
will  be  cancelled. 

The  content  of  each  transmitted  message  will  be  displayed  unobtrusive¬ 
ly  on  the  secondary  MFD  and  will  remain  displayed  until  replaced  by  the 
next  generated  speech  message. 

Priority  of  messages  to  be  transmitted  during  each  exercise  will  be 
preselected  using  a  data  entry  keyboard  associated  with  the  simulation  host 
computer.  Messages  with  priorities  not  selected  for  transmission  will  be 
presented  to  the  pilot  by  an  indicator  light,  annunciator,  or  audio  tone 
which  is  inherent  in  the  design  of  the  specific  aircraft  system  being 
simulated. 

The  following  alternative  output  message  control  schemes  will  be 
provided  in  the  simulation  design.  Each  alternative  will  be  selectable 
through  the  host  computer  keyboard. 

Alternative  4 1.  The  aircraft  audio  system  normally  will  handle  only 
radio  communications  unless  the  pilot  commands  differently.  In  the  event 
that  a  speech  generated  message  is  waiting,  the  word  "message"  will  be 
passed  over  the  audio  channel  when  no  radio  communications  are  present. 
When  convenient,  the  pilot  will  verbally  command  the  waiting  speech  message 
transmission  by  saying  "go  ahead"  to  the  speech  recognition  system. 

Alternative  42.  The  audio  system  normally  will  handle  only  radio 
communications.  In  the  event  that  a  systems  message  is  waiting,  the  words 
"see  MFD"  will  be  passed  over  the  audio  channel  when  no  radio  communica- 
itons  are  present.  The  message  will  then  be  displayed  on  the  secondary 
MFD, 


Alternative  ^3.  In  the  event  that  a  speech  generated  message  is  being 
transmitted  when  an  R/T  incoming  message  commences,  the  R/T  message  will  be 
recorded  and  stored.  It  will  then  be  played  back  to  the  pilot  when  the 
previous  speech  generated  message  is  complete.  The  playback  R/T  message 
will  assume  overriding  authority  in  the  queue. 
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Feedback 


Feedback  to  the  pilot  of  the  status  and  content  of  speech  system  input 
and  output  messages  will  be  made  as  follows: 

INPUT 


A  1/2  second  500  HZ  audio  tone  will  be  used  to  advise  the  pilot  that 
the  transmission  has  been  recognized  and  further  system  intelligence 
is  being  used  to  fulfill  the  command. 

Two  1/2  second  500  HZ  audio  tones  with  1/2  second  separation  will  be 
used  to  advise  that  the  command  has  been  fulfilled.  The  recognition 
of  the  message  will  be  displayed  unobtrusively  on  the  secondary  MFD  or 
displayed  on  a  separate  display  for  the  purpose  or  be  verbalized 
through  speech  generation. 

Mechanical  movement  of  appropriate  controls,  levers,  knobs,  etc.  will 
be  made  in  compliance  with  the  commanded  change  of  status  of  these 
devices. 

Illumination  of  lights,  movement  of  or  i  1 1  umi niation  of  annunciators 
will  be  made  in  compliance  to  the  commanded  change  of  status  of  these 
devices. 

In  the  event  that  the  input  is  not  recognized  by  the  system,  a  feed¬ 
back  message  of  "say  again"  will  be  generated  as  a  Priority  #1  mes¬ 
sage. 

OUTPUT 

Each  output  message  will  be  displayed  on  the  secondary  MFD  simul¬ 
taneously  with  the  speech  message  transmission  made  to  the  pilot. 

VOCABULARY 


The  speech  recognition  subsystem  will  be  speaker  dependent  and  use  an 
isolated  word  recognition  format.  An  example  of  a  speech  input  vocabulary 
is  given  in  Table  B-2.  Changes  to  the  speech  input  vocabulary  will  be 
accomplished  through  the  host  computer  keyboard. 

Two  types  of  generated  speech  outputs  will  be  provided- -stored  digi¬ 
tized  words  or  phoneme  synthesized  words.  Both  types  of  outputs  will  use 
a  male  sounding  voice.  The  speech  output  vocabulary  will  be  based  on 
individual  words  (in  contrast  to  expressions).  Individual  words  will  be 
combined  into  expressions  by  the  speech  generation  process.  The  speech 
output  vocabulary  is  shown  in  Table  8-3.  Use  of  digitized  or  synthesized 
outputs  will  be  by  a  keyboard  entry  into  the  host  simulation  computer. 
Changes  to  the  speech  output  vocabulary  will  be  accomplished  through  key¬ 
board  entry. 

Examples  of  the  speech  input,  output  and  interaction  expressions 
expected  to  be  used  for  the  simulation  are  shown  in  Tables  B-4  through  B-6. 
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TABLE  3-2.  INPUT  ISOLATED  WORD  VOCABULARY  FOR 
DEVELOPMENT  SIMULATION 

(NOTE:  Subject  to  Additions) 


"A" 

ALPHA 

AIR-TO-AIR 

AIR-TO-GROUND 

AIM 

AFT  -TAfK 

II  g  It 

BRAVO 

BEGIN 

BEGIN-CHECKLIST 


CHARLIE 

CHAFF 

CLIMB 

CLIMB -PROFILE 


CHANNEL 

CHANGE 


"D" 

DELTA 

DOWN 

DEGREES 

DISPLAY 

DESCENT 


EIGHT 

EIGHTY 

ENROUTE-NAV 


ECM 

ELEVEN 

EIGHTEEN 

EAST 


H  pti 

FOX  TROT 

FLARES 

FOURTY 

FORWARD-TANK 


FIFTEEN 

FOUR 

FIFTY 

FENCE-CHECKLIST 


FIVE 

FUEL-REMAINING 

FOURTEEN 


FORMATS 

FUEL 


HOTEL 

HUNDRED 

HEADING-TO-ALTERNATE 

HALT-CHECKLIST 

II  J  il 

INDIA 


TABLE  B-2.  INPUT  ISOLATED  WORD  VOCABULARY  FOR 
DEVELOPMENT  SIMULATION  (Continued) 


-  “J11 

J  JULIETT 


K  KILO 


L  LIMA 

LANTIRN-SEARCH 

LOAD-DP 

LOAD-FUEL 

LEFT 

-  "M" 

M  MIKE 

MASTER-ARM 

MODE 

MARK 

MINUS 

-  "N" 

N  NOVEMBER 

NINER 
NAV 

-  "0" 

0  OSCAR 

ON 
OFF 
ONE 
OVER 


P  PAPA 

POSITION 

POUNDS 

-  "Q" 

Q  QUEBEC 

HR" 

R  ROMEO 

RELEASE(D) 

RIGHT 

RESUME-CHECKLIST 


S  SIERRA 

SIXTY 
SEVEN 

I  SEVENTY 

!  STORES-REMAINING 


JTIDS-UPDATE 


LIGHTS 

LOAD 


NORTH 

NINETY 

NINETEEN 


POINT 

PANEL 

PLUS 


SIX 

SIXTEEN 

SEVENTEEN 

SOUTH 

STORES -CONFIGURATION 
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TABLE  B-2.  INPUT  ISOLATED  WORD  VOCABULARY  FOR 
DEVELOPMENT  SIMULATION  (Continued) 


T 

TANGO 

TWO 

TEN 

TWELVE 

TWENTY 

THIRTEEN 

THREE 

THIRTY 

TACAN 

TARGET( POSITION) 

THOUSAND 

-  "U" 

U 

UNIFORM 

UP 

UPDATE- DP 

UHF-FREQUENCY 

UHF-CHANNEL 

_  "v" 

V 

VICTOR 

VHF-FREQUENCY 

VHF-CHANNEL 

-  "W" 

w 

WHISKEY 

WEST 

-  "X" 

X 

X-RAY 

_  “Y“ 

Y 

YANKEE 

_  ,,2'1 

Z 

ZULU 

ZERO 

I 

I 

! 

i 

i. 
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TABLE  B-3.  DIGITALLY  STORED  WORD  OUTPUT  VOCABULARY 
FOR  DEVELOPMENT  SIMULATION 

(NOTE:  Subject  to  Additions) 


"A" 

A  ALPHA 
AIM 

AVIONICS 

AGAIN 

ALL 

IIQ  II 

B  BEE 
BRAVO 
BINGO 

lip  II 

C  CEE 
CHARLIE 
CAUTION 
COOLING 
CODE 
CONTROL 
CHECK 
CHANGE 
CONTROLS 
CLEAR 

"D" 

D  DEE 
DELTA 
DEGRADED 
DESCENT 
DO 

II  £11 

E  EEE 
ECHO 
EIGHT 
EIGHTY 
ELEVEN 
ELECTRICAL 
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TABLE  B-3.  DIGITALLY  STOREO  WORD  OUTPUT  VOCABULARY 
FOR  DEVELOPMENT  SIMULATION  (Continued) 


FENCE 


F  GOLF 

"H» 

H  HOTEL 
HANDS 
HIGH 
HILL 
HUNDRED 


I  I  INDIA 

{  "J" 

J  JAY 

i  JULIETT 


K  KAY 
KILO 

llj^ll 

L  LIMA 
LOW 
LEFT 

"M" 

M  MIKE 
MASKING 
MASTER 

"N" 

N  NOVEMBER 
NINER 
NINETY 
NAV 
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TABLE  B-3.  DIGITALLY  STORED  WORD  OUTPUT  VOCABULARY 
FOR  DEVELOPMENT  SIMULATION  (Continued) 


"O" 

0  OSCAR 
ONE 
ON 
OFF 
OF 

«»p»» 

P  PAPA 
POUND 
POWER 


Q  QUEBEC 

iir  ii 

R  ROMEO 
RIGHT 
RUDDER 
REJECT 

"S'* 

S  SIERRA 
SIX 
SIXTY 
SEVEN 
SEVENTY 
SYSTEM 
SAY 
SEE 


T  TANGO 
TWO 

TWENTY 

THREE 

THIRTY 

TWELVE 

THREAT 

THOUSAND 

TARGET 

TOP 

TURN 

TREE 

TO 

TEN 

"U" 

U  UNIFORM 
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TABLE  B-3.  DIGITALLY  STORED  WORD  OUTPUT  VOCABULARY 
FOR  DEVELOPMENT  SIMULATION  (Continued) 


"V" 

V  VICTOR 
"W" 

W  WHISKEY 
WANT 

"Y- 

Y  YANKEE 
YOU 

11^  It 

Z  ZULU 
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TABLE  B-4.  DESIRED  SIMULATION  INPUT  EXPRESSIONS 


PRIMARY  EXPRESSION 

ALTERNATE  EXPRESSION 

Chaff  or 

Flares 

Release 

Chaff 

Flares 

Release  Chaff 

Release  Flares 

Radio 

Channel  or 

Frequency 

Change 

VHF  One  One 

UHF  Niner 

TACAN  Five  Six 

VHF  One  Three  Six  Point 
Seven  Zero 

UHF  Three  Zero  One 

Point  One 

VHF  Channel  One  One 

UHF  Channel  Niner 

TACAN  Channel  Five  Six 

VH.  Frequency  One  Three 
Six  Point  Seven  Zero 
UHF  Frequency  Three  Zero 
One  Point  One 

System 

Mode 

Selection 

ECM 

Master  Arm 

Lanti rn  Search 

Air-to-Air 

Air-to-Ground 

MFD  Formats 

Etc. 

ECM  On  ! 

Master  Arm  On 

Lanti rn  Search  Mode 
Air-to-Air  Mode 

Air  to  Ground  Mode 

Etc. 
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TABLE  B-4.  DESIRED  SIMULATION  INPUT  EXPRESSIONS  (Continued) 


Primary  Expression 

Alternate  Expression 

Interior  and 
Exterior 

Lighting 

Control 

Panel  Down 

Panel  Up 

Panel  On 

Panel  Off 

Position  Up 

Position  Down 

Position  On 

Position  Off 

NAV  On 

NAV  Off 

Panel  Lights  Down 

Panel  Lights  Up 

Panel  Lights  On 

Panel  Lights  Off 

Position  Lights  Up 
Position  Lights  Down 
Position  Lights  On 
Position  Lights  Off 

NAV  Lights  On 

NAV  Lights  Off 

Insertion 
of  Data  Into 
Systems 
(Permission 

Or  In  Flight) 

- -  - 

D.P.  One-North  Three 
Three  Degrees  Two 

Niner  Point  Five, 

West  One  Two  One 

Degrees  Five  Seven 

Point  Two 

Load  D.P.  One,  North — 

D.P.  Niner  - 

Load  D.P.  Niner - 

Aim  Niner  C,  Two 

Left,  Two  Right 

Load  Aim  Niner  C 

Two  Left,  Two  Right 

I 

Fuel,  Forward  Tank 

Zero,  AFT  Tank 

Ten  Thousand 

Load  Fuel ,  Forward 

Tank  Zero,  AFT 

Tank  Ten  Thousand 

Pounds 
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TABLE  B-4.  DESIRED  SIMULATION  INPUT  EXPRESSIONS  (Continued) 


PRIMARY  EXPRESSION 


ALTERNATE  EXPRESSION 


Display  of 
Data  On 
MFD's 


Display  Right  - 

Display  Left  - 

Display  Change  (Left  to 
Right  to 
Left) 


Display  Right  MFD  - 

Display  Left  MFD  - 


(Typical  Format  Callout) 


(Typical  Format  Callout) 


Enroute 

Target 

Cl  imb 

JTIDS 

Stores 

Fence 


Enroute  NAV 
Target  Position 
Climb  Profile 
JTIDS  Update 
Stores  Configuration 
Fence  Checklist 


D.P.  One,  (Pause)  Mark 


Update 

Navigation 

Position 


Update  DP  one  (Pause)  j 
Mark  I 


I 


D.P.  Niner  (Pause)  Mark 


Update  D.P. ,  Niner  i 

(Pause)  Mark  j 

i 

t 


1 


Request 

Critical 

System 

Information 


Fuel  Remaining 
Stores  Remaining 
Heading  to  Alternate 

Air  Start  Checklist 
- Checklist 


Etc. 


Say  Fuel  Remaining 
Say  Stores  Remaining 
Say  Heading  to  Alter¬ 
nate  ' 

Say  Airstart  Checklist  ! 

Say - Checklist  ! 

Display  --  Checklist 
Etc.  ! 
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TABLE  B-5.  DESIRED  SIMULATION  OUTPUT  EXPRESSIONS 


PRIMARY  EXPRESSION 

ALTERNATE  EXPRESSION 

SA  Ten  One  O'clock 

Threat 

Warnings 

SA  Ten  Twelve  O'clock 

SA  Twenty  One  O'clock 

Preceed  all  messages 
with  the  word  "THREAT" 

SA  Twenty  Twelve  O'clock 

Unidentified  One  O'clock 

Unidentified  Twelve  O’Cock 

NOTE:  Relative  Bearing 
may  be  followed  by  "High" 
or  "Low" 

SECURE  CODE 
CHANGES 

IFF  CODE  TWO  CEE 

ETC. 

CHANGE  IFF  fQDE  TO 

TWO  rr: 

ETC. 

Master 

Warning  & 
Caution 

Caution  On 

Caution  Avionics  Cooling 

Master  Caution  On 
Master  Caution 
Avionics  Cooling 

Caution  - 

Etc. 

Master  Caution - 

Etc. 

l 
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TABLE  B-5.  DESIRED  SIMULATION  OUTPUT  EXPRESSIONS  (Continued) 


Primary  Expression 


Alternate  Expression 


Time 

Related 

Action 

Prompting 


Update  NAV 
Fence  Check 
Change  IFF 
Top  of  Descent 
Etc. 


Aerodynamic 

Performance 

Envelope 

Exceedance 


Power  Off,  Left  Rudder 
Power  Off,  Right  Rudder 
Hands  Off  Controls 
Etc. 


(See  Table  F4 

Verbal  Input  Vocabulary) 

Recogni tion 
of  All  Speech 
Inputs 


(See  Table  F4 
Input  Vocabulary) 


Speech 

System 

Control 


Say  Again 
Do  You  Want 
Reject 
See  M.F.D. 


TABLE  B-5.  DESIRED  SIMULATION  OUTPUT  EXPRESSIONS  (Continued) 


PRIMARY  EXPRESSION 

ALTERNATE  EXPRESSION 

Critical 

Fuel 

Remaining 

Bingo  Fuel 

Altitude 

Read  Out 

Ten  Thousand 

(Every  1000  Ft.) 

One  Thousand 

(Every  100  Ft.) 

One  Hundred 

(Every  10  Ft.) 

Fifty 

Ten  Thousand  Feet 

(Every  1000  Ft. 

One  Thousand  Feet 

(Every  100  Ft.) 

One  Hundred  Feet 

(Every  10  Ft.) 

Fifty  Feet 

Mission 

Critical 

System 

Degredation 

Degraded  Threat  Warning 
Degraded  ECM  System 
Degraded  NAV  System 
Degraded  Flight  Control 
Degraded  Electrial  Power 
Deqraded  (Two  Words) 

Etc. 

Laser 

Designated 

Target 

Masking 

Masking 

Target  Masking 
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TABLE  B-6.  DESIRED  OF  SIMULATION  INTERACTIVE 
SPEECH  EXPRESSION 


INPUT  EXPRESSION 

OUTPUT  EXPRESSION 

Requests  for 
System  Data 

Fuel  Home 

Fuel  to  Alternate 

Three  Five  Zero  Zero 
(NOTE:  Digit  Expression) 

Request  for 
Status  of 
Automatic, 

Semi  Automatic 
or  Manual 
Checking 
Procedures 

Fence  Check  Status 

Threat  Status 

Status 

(NOTE:  Input  Expression 
uses  up  to  two  words 
in  front  of  "STATUS") 

Turn  Master  Arm  On 

All  Clear 

(NOTE:  Output  Expression 
can  contain  up  to  five 
words  from  specialized 
vocabulary) 

Request  for 
Flight  Plan 

Data 

Features  for  D.P.  Three 
Target  One  Elevation 

(NOTE:  Input  Expression 
uses  up  to  five  words) 

One  Tree  Hill 

Two  Five  One  Feet 

(NOTE:  Output  Expression 
can  contain  up  to  five 
words) 

Request 

Remedial 
Information 
on  Degraded 
System 

Operation 

Fuel  Control  Fix 

(NOTE:  Input  Expression 
uses  up  to  five  words) 

Switch  to  BEE,  U,  CEE 

(NOTE:  Output  Expression 
can  contain  up  to  five 
words) 

(BEE  U  CEE  =  Backup  Control) 
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SYNTAX 


Functional  and  situation  discrimination  logic  (SYNTAX)  will  conform  to 
the  generalized  "granmar"  shown  in  Table  B-7.  The  features  of  this  grammar 
are  summarized  as  follows: 

o  An  "utterance”  is  either  a  function,  an  action  or  an  expression. 

o  Functions  are,  in  effect,  single  word  commands.  In  some  cases, 
several  individual  words  are  strung  together  to  yield  a  single  word 
conmand  that  would  be  recognizable  by  the  isolated  word  recognition 
system. 

o  Actions  are  two  word  commands  that  identify  a  "device"  to  be  acted 
on  and  a  “state"  to  which  it  is  to  be  changed.  Two  isolated  recog¬ 
nitions  are  therefore  required. 

o  Expressions  are  multiple  word  commands  that  require  more  than  two 
isolated  recognitions.  Typically  the  additional  words  are  required 
in  order  to  express  a  quantity  consisting  of  one  or  more  digits 
with  (optionally)  an  associated  set  of  units  and  a  “direction"  (or 
"destination"). 

o  Because  of  the  nature  of  the  items  that  might  be  referred  to  in  an 
expression,  it  is  necessary  to  have  a  terminator  (e.g. ,  the  word 
"OVER").  The  terminator  is  not  necessary  for  either  functions  or 
actions. 

The  granular  has  been  designed  for  simple  user  operation.  No  distinc¬ 
tions  have  been  made  to  insure  that  every  syntactically  correct  statement 
is  also  semantically  correct.  Although  the  structure  permits  many  (in 
fact,  an  infinite  number  of)  semantically  incorrect  statements,  at  the  same 
time  there  is  a  greater  number  of  ways  to  utter  a  semantically  correct 
statement.  This  is  both  good  and  bad--good  because  it  gives  the  operator 
the  ability  to  request  the  same  information  in  several  different  ways,  and 
bad  because  the  operator  must  have  a  mental  model  of  what  consititutes  a 
correct  statement  and  what  does  not.  This  flexibility  has  been  allowed  so 
as,  first,  to  capture  both  the  primary  forms  and  the  alternate  forms  in  the 
input  phrase  repetoire  with  a  single  grammar  and,  second,  because  the 
functions  that  are  required  need  it.  If  desired,  the  syntax  can  be  revised 
so  that  the  structure  is  less  flexible.  However,  the  problem  will  remain 
that  a  syntactically  correct  sentence  may  be  "physically  unrealizable" 
e.g.,  the  operator  may  try  to  "SELECT  VHF -CHANNEL  ZERO  POINT  ZERO  THREE 
TWO."  Software  to  support  the  speech  input  system  will  have  to  provide 
adequate  checks  for  such  situations. 

TRAINING  THE  SYSTEM 


Each  system  user  will  "train"  the  speech  recognition  subsystem  off¬ 
line  using  multiple  iteration  of  the  same  words  spoken  with  different 
emphasis  and  intonation.  Although  further  research  on  this  issue  is  re¬ 
quired,  preliminary  indications  are  that  speech  variability  during  template 
creation  may  lead  to  better  recognition  accuracy  during  operations. 
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TABLE  8-7.  PROPOSED  SPEECH  INPUT  SYNTAX 


UTTERANCE  function  |  action  |  expression 

FUNCTION  -  CEAJ?  |  FLARES  |  ECM  |  MASTER -ARM  1  LAND  IRK- S EAEC E  | 

A ZR -TO-GROUND  |  AIR-TO-AIR  |  ...  I  FUEL -REMAIN INC  } 
STORES -REMAINING  |  EEADING-TO-ALTERNAIE  |  CLIMB  |  DESCENT 

ACTION  device  +  state  |  verb  +  function 

DEVICE  function  !  PANEL (LIGETS )  |  POSITION (LI GETS )' | 

NAV  (LIGETS  ) 

STATE  -  UP  |  DOWN  |  OFF  |  ON  |  MODE  |  RELEASED 

VERS  BEGIN  |  RELEASE  |  BEGIN -CHECKLIST  |  EALT -CHECKLIST  | 

RZSEME-CEECKLIST 

EXPRESSION::*  systen  -r  quantity  +  OVER 

SYSTEM  VEF -FREQUENCE  |  VE? -CHANNEL  |  UEF -FREQ3ENCY  |  UHF -CHANNEL  I 

TACAN  |  AIM  |  (LOAD)-D?  |  (LOAD )-FUEL  |  DISPLAY  | 

UPDATE-DP  | 

QUANTITY  nunber  [units]  [direction]  [quantity]  !  format  [direction] 

NUMBER  -  ZERO  |  ONE  |  TWO  |  THREE  |  FOUR  \  FIVE  |  SET.  |  SEVEN  | 

'EIGHT  |  NINE?,  j  TEN  |  ELEVEN  |  TWELVE  |  THIRTEEN  I 
FOURTEEN  |  FIFTEEN  j  SI2TEEN  |  SEVENTEEN  |  EIGHTEEN  | 
'■TNETEEN  |  TWENTY  1  THIRTY  |  FORTY  1  FIFTY  |  SIXTY  | 
SEVENTY  1  EIGHTY  j  NINETY  |  HUNDRED  |  THOUSAND  | 

PLUS  |  MINUS  |  POINT 

DIRECTION  -  NORTE  |  SOUTH  |  EAST  |  WEST  |  RIGHT  I  LEFT  |  MARK  1 
FORWARD  (TANK)  |  AFT  (TANK) 

UNITS  : :  -  DEGREES  |  POUNDS  I  ALPHA  j  BRAVO  |  CHARLIE  ]  . . .  j  ZULU  | 


FORMAT  : :  ■  ENROUTE  (NAV)  j  TARGET -(PCS  IT  ION)  |  CLIMB- (PROFILE  )  | 

JT IDS - (UPDATE )  |  STORES -(CONFIGURATION)  i  FENCE- (CHECKLIST) 


NOTE;  Parentheses  indicate  a  specific  examDle  and  brackets  indicate  a 
generic  category. 
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SIGNING-ON 

Prior  to  using  the  system,  the  user  will  sign-on  through  the  host 
simulation  computer  keyboard. 
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Canyon  Research  Group,  (nc; 


H 


m 


Canyon  Research  Group,  Inc. 
and 

Honeywell,  SRC 


THE  APPLICATION  OF  SPEECH  I/O  AS  AN  ALTERNATE 
MAN/MACHINE  INTERFACE  FOR  FUTURE 
TACTICAL  AIRCRAFT  COCKPIT  DESIGN 

BACKGROUND 

The  Canyon/Honeywell  team  is  engaged  on  an  R4D  contract  with  the  Air 
Force  Wright  Aeronautical  Laboratory  (AFWAL/FIGR)  to  use  speech  as  an 
alternate  input/output  (I/O)  mode  for  eyes-out,  hands-on  throttle  and  stick 
(HOTAS)  operations  in  the  single- seat  fighter  aircraft.  Because  you  are 
part  of  the  pilot  community  which  will  use  such  new  system  concepts,  we  are 
soliciting  your  opinion  on  many  aspects  of  the  proposed  design. 

The  concept  is  that  you  will  speak  to  the  "speech  system"  and  command 
it  to  implement  switch  manipulations,  request  information  from  other 
aircraft  systems,  etc.  The  speech  system  will  recognize  your  voice,  react 
to  your  conmand  and  then  advise  you  that  the  command  has  been  fulfilled. 
Although  the  speech  system  may  talk  to  you  in  response  to  your  commands, 
R/T  will  always  be  given  priority.  In  fact,  you  will  be  able  to  turn  the 
speech  output  "off"  at  which  time  feedback  about  your  speech  commands  could 
be  shown  on  a  multi-function  display. 

In  the  near-term  cockpit  designs  (1985-1990),  we  anticipate  that  only 
some  important  functions  (i.e.,  threat  management,  weapons  control  and 
certain  emergency  functions)  will  be  implemented  with  speech  I/O  as  an 
alternative  (optional)  mode  of  input. 


741  lakefieid  Road,  Suita  B,  Westlaka  Village,  California  91361  •  (213)  889-5072 
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One  design  option  for  the  far-term  cockpit  design  (1990-2000)  is  to 
provide  the  pilot  with  an  optional  speech  1/0  mode  for  nearly  all  cockpit 
functions,  with  the  exception  of  throttle  and  stick  control.  This  is  an 
entirely  new  concept  for  cockpit  design,  and  it  implies  some  new  approaches 
to  the  functional  layout  of  panel  space.  System  control  would  become  more 
automated,  but  the  back-up  mode  for  all  systems  would  rely  heavily  on 
manual  manipulation  or  speech  1/0,  whichever  is  more  convenient  for  the 
pilot  at  that  time. 

THIS  QUESTIONNAIRE 

The  following  questions  are  to  provide  you  with  an  opportunity  to 
contribute  your  ideas  and  opinions  about  the  best  ways  to  apply  speech  1/0 
in  the  fighter  cockpit  of  the  future.  Your  ideas  are  important.  The 
information  obtained  from  this  questionnaire  will  be  a  primary  factor  in 
the  cockpit  design  recommendations  which  will  be  submitted  to  the  Air  Force 
R&D  community. 

The  personnel  data  (organization,  name,  experience,  etc.)  are  for 
purposes  of  data  analysis  only.  The  information  will  be  kept  confidential. 
No  answers  or  comments  will  be  associated  with  your  name. 

Please  take  the  time  to  answer  the  questions  carefully.  Give  comments 
and  examples  on  the  back  of  the  page  if  you  have  time. 

The  6  point  scales  provided  after  each  question  represent  a  continuum 
of  usefulness,  not  useful  to  very  useful.  The  mid-points  of  this  scale 
could  be  described  as  moderately  useful/not  useful,  and  in  the  middle, 
sightly  useful/not  useful.  You  should  indicate  your  opinion  on  one  of  the 
6  appropriate  vertical  line  markers  and  not  in-between  markers. 


PERSONNEL  DATA 


Name/Rank  _  Date _ 

Organization _  Job/Position _ 

Age  _  No.  Years  Military  Experience 

Pilot  Experience:  Please  list  approximate  hours  by  type 
Aircraft  Type  Hours 


Computer  Experience:  (check  those  that  apply) 

a)  Major _  b)  Courses  _  c)  own  home  computer _ d)  other 


Other  comments  on  attitudes,  education  or  experience  that  might  influence 
your  answers  to  this  questionnaire: 


1.  A  voice  generator  could  be  linked  with  a  Ground  Proximity  Warning 
System  providing  specific  voice  altitude  readouts  below  "given"  or 
"pre-set"  altitudes. 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  _ _ , _ _ _ , _ , _ » v*ry 

Useful  Useful 


2.  The  voice  generator  system  could  provide  exact  threat  information 
details  such  as  the  type,  direction  and  priority  level;  for  example, 
"S  A  10  Launch,  4  O'clock." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not 

Useful 


Very 

Useful 


3.  The  voice  generator  system  could  be  used  to  give  the  pilot  reports  of 
new  threats;  for  example,  "New  guy.  Quad  two." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not 

Useful 


t- 


4 


Very 

Useful 


4.  The  voice  system  could  report  the  position  and  bearing  of  a  bogey,  for 
example,  "Threat  at  two  O'clock  high." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not 

Useful 


Very 

Useful 
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5.  A  voice  message  could  inform  you  when  the  system  has  reached,  "Bingo 
fuel." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  i - 1 - 1 _ 1 _ i _ i  Very 

Useful  Useful 


6.  A  voice  message  could  remind  the  pilot  of  fence-check  when  20  minutes 
out. 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  ' - 1 - - - 1 - 1 _ i _ i  Very 

Useful  Useful 


7.  The  voice  generator  could  inform  the  pilot  when  the  temperature  of  the 
engine  had  reached  over-heat  limits. 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  i — - 1 - 1 _ i _ _ _ i  Very 

Useful  Useful 


8.  The  voice  generator  could  detect  and  report  degradation  of  the  threat 
warning  system. 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  l 
Useful 


Very 

Useful 


9.  The  voice  system  could  report  an  abnormal  fuel  distribution  situation. 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  i _ i _ i _ i _ i _ i  Very 

Useful  Useful 


10.  A  voice  message  could  prompt  when  certain  tasks/items  should  be 
performed,  for  example,  "Change  IFF  code." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  t _ i _ i _ i _ i _ i  Very 

Useful  Useful 


11.  The  voice  generator  could  be  linked  with  mode  changes  to  provide 

additional  feedback  for  such  confirmatory  changes  as  "TA/TF,  engaged," 
or  "DP3  updated." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  \ _ I _ l _ i _ i _ i  Very 

Useful  Useful 


12.  The  voice  system  could  remind  the  pilot  of  critical  points  for 
pull-out. 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  l _ I _ l _ l _ i - 1  Very 

Useful  Useful 
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13.  When  requested,  the  voice  system  could  announce  the  current  ECM  mode. 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  i _ i _ i _ l _ i _ i  Very 

Useful  Useful 


14.  The  voice  generator  could  report  hung  ordnance  information. 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  i _ i _ l _ i _ i _ i  Very 

Useful  Useful 


15.  The  voice  generator  could  inform  the  pilot  when  the  NAV  system  has 
drifted  beyond  acceptable  limits. 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  | _ l _ l _ i _ i _ l  Very 

Useful  Useful 


16.  The  voice  generator  could  provide  the  pilot  with  a  solution  or 

description  to  a  problem  caused  by  a  degraded  system;  for  example,  if 
INS  has  large  errors,  the  GPS  should  be  used  instead. 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  i _ 1 _ | _ I _ I _ I  Very 

Useful  Useful 


The  voice  system  could  verbally  enunciate  master  caution  panel 
occurrences. 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  i_ 
Useful 


Very 

Useful 


18.  The  voice  generator  should  tell  the  pilot  when  the  ECM  is  locked-on, 
especially  when  the  pilot's  eyes  are  outside  the  cockpit. 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  u. 
Useful 


Very 

Useful 


19.  The  voice  generator  could  inform  where  the  bogey  is  by  quadrant,  or  by 
"clock"  indications. 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  L. 
Useful 


Very 

Useful 


The  voice  generation  unit  could  inform  the  pilot  when  the  life  of  a 
flight  control  battery  is  getting  low,  below  some  preselected  level. 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  L 
Useful 


Very 

Useful 
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21.  During  a  violent  spin,  the  voice  system  could  say  which  direction  the 
plane  is  turning/spinning. 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  i _ i _ i _ i - 1 - 1  Very 

Useful  Useful 


22.  The  voice  system  could  enunicate  the  center  of  gravity  movement  on  an 
air  refueling  operation. 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  | _ i _ 1 _ i - _J - 'Very 

Useful  Useful 


23.  Assuming  that  laser-guided  bombs  are  in  use,  speech  generation  could 
provide  warnings  to  a  pilot  that  he  was  about  to  mask  a  laser 
designation  with  a  particular  part  of  the  vehicle. 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  i _ i _ i - 1 - 1 - » Very , 

Useful  Useful 
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QUESTIONS 
PART  II 


These  questions  address  the  role  that  speech  recognition  or  command  could 
play  in  the  fighter  cockpit.  Speech  recognition  will  involve  speaker 
dependent,  (individually  trained  and  stored)  vocabulary  items.  The  format 
for  speaking,  to  the  speech  'box'  will  use  individual  words  or  2  -  3  word 
phrases.  Appropriate  vocabulary  words  will  be  selected  and  used  in 
succinct,  reliably  recognized  phrases.  Feedback  to  the  pilot,  indicating 
that  the  desired  input  was  received,  could  be  given  either  visually  or 
auditorily. 
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24.  The  deployment  of  CHAFF/FLARES  could  be  performed  by  voice  command, 
for  example,  by  speaking  "Release  CHAFF,"  or  "Release  FLARES." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not 

Useful 


4 _ i _ i _ i _ t  Very 

Useful 


25.  A  voice  command  could  be  used  to  jettison  the  stores  when  necessary, 
for  example,  a  verbal  command  "Drop  stores." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  , _ , _ , _ , _ , _ t  Very 

Useful  Useful 


26.  A  voice  cotmand  could  be  used  to  access  the  air  terminal  information 
service  (ATIS)  in  the  spoken  form,  "ATIS  Check." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not 

Useful 


Very 

Useful 


27.  The  voice  command  could  be  used  to  access  JTIDS  information  as  a 
spoken  request,  "Display  JTIDS." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  . _ _ _ . _ _ _ 1 _ \ 

Useful  Useful 
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28.  A  voice  command  could  call  up  the  approach  plate  for  suitable  display 
access  for  various  locations;  for  example,  "Display  Nellis  Approach 
Two  One." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  I _ I _ 1 _ 1 _ i _ i  Very 

Useful  Useful 


29.  A  voice  command  could  link  the  mission  planning  and  review 
information,  as  a  spoken  command,  “Mission  Review." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  l _ 1 _ l _ i _ l _ l  Very 

Useful  Useful 


30.  A  voice  command  system  could  change  or  be  used  to  select  various 
displays  on  the  F-16C  MFD;  for  example,  "Display  Radar  Left." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  « _ i _ i _ i _ i _ l  Very 

Useful  Useful 


31.  A  voice  command  could  be  used  to  turn  the  ECM  POD  on;  for  example, 
"Activate  ACM." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  « _ i _ i _ i _ i _ i  Very 

Useful  Useful 


133 


i 


32.  A  voice  utterance  could  be  used  to  open  and  close  the  air  refueling 
doors;  for  example,  “Open  Refuel  Door." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  _ _ i _ i - . - . - iVery 

Useful  Useful 


33.  A  voice  recognition  system  could  initiate  UHF/VHF  radio  channel 
charges  in  the  form,  "Select  VHF,  one-two-one-point-five." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  4 _ , _ , _ , _ .  Very 

Useful  Useful 


34.  A  voice  cormand  could  also  make  TACAN  channel  changes  by  using  a 

series  of  isolated  word  utterances;  for  example,  "Change  TACAN"  .  .  . 
"eighty- six"  or  "Change  TACAN,  PANOCHE." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  , _ , _ i _ , _ i _ I  very 

Useful  Useful 


35.  A  voice  recognition  system  could  be  used  to  input  or  set  pre-flight 
data  information  concerning  route,  weapons,  fuel,  etc. 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not 

Useful 


x  Very 
Useful 
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36.  Many  of  the  FCNP  keystrokes  could  be  performed  by  a  voice  recognition 
system  for  such  commands  as  the  NAV  coordinates  or  updates  by  saying, 
"Lat.  North  niner  degrees  three  two  point  eight  ..." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not 

Useful 


i 


.  Very 
Useful 


37.  The  voice  recognition  system  could  be  used  to  change  any  mode 

selections  during  flight;  for  example,  "Select  Air-to-Air,“  “Select 
Dive  Toss." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  , _ , _ j _ , _ , _ ,  Very 

Useful  Useful 


38.  The  voice  recognition  system  could  interact  with  radar  controls  to 

change  such  parameters  as  the  range,  bar  scan  pattern,  AL-EL  coverage, 
intensity,  etc. 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  , _ , _ , _ , _ , _ ,  Very 

Useful  Useful 


39.  The  voice  command  system  could  be  interfaced  with  the  stores 
management  system  by  the  spoken  commands,  "Select  Master  Arm." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  , _ , _ , _ , _ i  i  Very 

Useful  Useful 
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40.  A  verbal  command  request  could  present  the  pilot  with  the  status  of 

any  A/C  system  during  flight  (the  information  could  be  diagnostic  if  a 
fault  exist). 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  < _ i _ i _ i— - 1 - »  Very 

Useful  Useful 


41.  A  voice  command  could  be  used  to  enunciate  or  display  critical  items 
before  entering  FEBA;  for  example,  “Fence  Check." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  i _ l _ i _ l _ _ _ 1 - 1  Very 

Useful  Useful 


42.  A  voice  command  could  be  used  to  request  aid  in  endurance  calculation 
by  the  spoken  utterance,  “Display  Endurance,"  or  "Say  Endurance." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  , _ i _ i _ - _ i - 1 - -J  very 

Useful  Useful 


43.  A  voice  command  could  be  used  to  update  the  pilot  on  a  flyover  by 
speaking;  for  example,  "Mark  Position." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  i _ i _ i _ _ _ i _ i - 1  Very 

Useful  Useful 
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44.  A  voice  command  could  be  used  to  call  up  individual  MFD  control  pages; 
for  example,  "Stores  Management,  Left." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  i _ i _ i _ i _ i _ i  Very 

Useful  Useful 


45.  A  voice  conmand  system  could  perform  changes  in  levels  of  interior  and 
exterior  lighting;  for  example,  by  speaking,  “Dim  Interior  Lights,"  or 
"Kill  Exterior  Lights." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  i _ I _ i _ 1 _ i - 1  Very 

Useful  Useful 


46.  A  voice  command  system  could  call  up  various  checklists  and  have  the 
appropriate  one  enunciated  or  displayed;  for  example,  "Report  Air 
start  checklist,"  or  "Display  Air  start  checklists." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  | _ i _ i _ i _ i _ l  Very 

Useful  Useful 


47.  A  voice  conmand  could  be  used  in  weapon  deployment;  for  example,  with 
a  verbal  "Go  slewable,"  thus  activating  target  designator  movement. 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  i _ I _ I _ i _ i _ l  Very 

Useful  Useful 


137 


l 


1 


48.  A  voice  command  could  be  interface  with  a  LANTIRN  system  to  activate 
delivery  modes,  weapon  selection,  and  step  from  target-to-target;  for 
example,  "Step  target.  Pod  brighter." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  i  i 

Useful 


j _ i  Very 

Useful 


49.  Voice  command  could  be  used  to  assist  in  missile  control , ^especi al ly 
in  a  track-while-scan  mode;  for  example,  "Cage,"  "Uncage." 


How  useful  would  it  be  to  perform  this  task  by  speech? 


QUESTIONS 
PART  III 


These  questions  address  the  potential  role  that  speech  recognition  and  speech 
generation  could  play  together,  in  an  interactive  dialogue  format.  Many  of 
the  preceding  voice  command  task  examples  could  be  backed  up  with  speech 
generation  feedback.  However,  the  following  suggested  tasks  extend  the  voice- 
interactive  capability  to  cover  a  more  unique  role  or  capability  for  the  two 
technologies  to  be  implemented  simultaneously.  The  'dialogues’  will  use 
only  two  or  three  interactions  involving  speech  input  and  speech  output, 
in  short  transacti ons ,  avoiding  elaborate  conversations. 
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50.  A  voice  interactive  dialogue  capability  would  allow  the  pilot  to  ask 
for  “fuel  remaining"  by  voice  command  and  receive  the  appropriate 
figure  through  voice  generation  (this  may  be  appropriate  during  poor 
night  light  conditions) 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not  _ _ » _ i _ i _ l _ i  Very 

Useful  Useful 


51.  A  voice  interactive  system  should  allow  the  pilot  to  ask  for  "Bingo 
fuel,"  and  receive  the  reply  through  speech  generation. 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not 

Useful 


Very 

Useful 


52.  A  voice-interactive  system  could  allow  the  pilot  to  ask  for  "fuel 
home"  and  receive  the  replay  verbally  through  speech  generation. 


How  useful  would  it  be  to  perform  this  task  by  speech? 


Not 

Useful 


Very 

Useful 


53.  A  voice  interactive  dialogue  system  could  be  used  to  call  up  a  future 
destination  point  (DP)  and  then  request  a  particular  ground  feature, 
the  response  to  which  may  be  verbal  in  the  form: 

Pilot:  “D.P.  three"  System  "One-tree  hill" 

Pilot:  "Distance:  System  "Two-zero  miles" 

How  useful  would  it  be  to  perform  this  task  by  speech? 

Not  l 
Useful 

i 


Very 

Useful 


SUMMARY  QUESTIONS  AND  COMMENTS 


54.  Assume  you  had  a  speech  I/O  system  in  an  eyes-out/hands-on-stick  and 
throttle  (HOTAS)  situation.  Please  rate  how  helpful  speech  I/O  might 
be  in  the  following  functional  areas:  (check  one  of  the  three  blanks 
on  each  line) 

VERY  POSSIBLY  NOT 

HELPFUL  HELPFUL  HELPFUL 


Vehicle  Control 
Navigation 
Fire  Control 
Subsystem  Management 
Threat  Management 
Weapons  Delivery 
Target  Acquisition 
R/T  Communications 


COMMENTS:  Please  comment  and  give  examples  for  as  many  of  these  eight 

categories  as  you  can.  (use  the  back  of  the  pages  for  extra 
space) 
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55.  The  pilot  will  need  confirmation  that  appropriate  action  has  been 
taken  for  each  speech  command  that  he  gives.  Sometimes  the  feedback 
will  be  obvious,  as  in  a  display  mode  change.  When  feedback  is  NOT 

obvious,  it  can  be  given  either  by  speech  output  or  it  can  be 
displayed  on  an  MFD,  or  both.  Please  comment  (give  examples,  if 
possible)  on  how  you  would  like  to  get  confirmation  of  speech  command 
inputs:  (consider  (1)  speech  output,  (2)  displayed  information,  and 
(3)  both) 


56.  As  previously  indicated,  R/T  will  have  priority  on  the  audio  channel . 
One  suggested  method  for  prioritizing  the  speech  system  outputs  is  as 
fol  1 ows : 


Priority  #1  -  Ultra  important  message  which  assists  in  the  safe 

conduct  of  the  mission.  These  messages  will  override 
any  other  message  being  transmitted  by  the  speech 
generation  system. 

Priority  #2  -  Very  important  messages  which  assist  in  the 

management  of  the  extraordinary  events  experienced 
during  a  mission,  e.g.,  flight  control  system  servo 
failure. 

Priority  #3  -  Important  messages  which  assist  in  the  management  of 

ordinary  events  experienced  in  flight,  e.g., 
endurance/fuel  remaining. 

Priority  #4  -  Routine  messages  which  assist  the  pilot  in  routine 

work,  e.g.,  reciting  checklists. 


Each  priority  #2  through  #4  speech  message  will  be  completed  before 
the  next  message  is  commenced,  unless  a  priority  number  one  message 
intervenes.  When  more  than  one  message  is  in  the  que,  the  priority  will  be 
adjusted  before  the  next  message  is  transmitted.  Incomplete  messages  will 
be  reassigned  to  their  correct  priority  position  and  retransmitted  as 
complete  messages.  Messages  which  are  delayed  more  than  20  seconds  by  the 
priority  rules  will  not  be  transmitted  and  will  be  cancelled. 
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Do  you  agree  with  this  priority  concept? 

YES  _  NO  _ 

COMMENTS/SUGGESTIONS:  _ 


57.  Please  add  any  other  comments,  general  or  specific,  on  speech  I/O 
applications  in  future  tactical  aircraft. 


background 


distributed  to  over  90  pilots  at 
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NELLIS  AFB  F-16  PILOT  QUESTIONNAIRE  COMMENTS 


PERSONNEL  DATA: 


Pi 

lot 

#  9: 

Pi 

lot 

#14: 

Pi 

lot 

#15: 

Pi 

lot 

#16: 

Pi 

lot 

#24: 

Pi 

lot 

#25: 

Pilot 

#27: 

Pi 

lot 

#37: 

Pi 

lot 

#41: 

Pi 

lot 

#44: 

QUESTION 

#1 

Observed  demeonstration  of  voice  commanded  Stores  Management 
System  (SMS)  at  Hill  AFB,  UT  in  May  82. 

Wrote  concepts  for  future  weapons  systems  at  TAC  HQ  1978-80. 

Masters,  aerospace  engineering. 

Engineering  operations  major. 

Electronic  warfare  off icer/navi gator  for  3  years. 

Not  really,  except  very  aware  of  computer  capabilities  and 
current  technology. 

Career  fighter  pilot;  B.S.,  Physics;  M.S.  in  Military  Sci¬ 
ence,  nearly  completed  M.S.  in  Information  System  Theory. 

Elec,  engineer  major. 

BSEE  with  gcod  familiarity  of  computers. 

1120  combat  hours. 


A  voice  generator  could  be  linked  with  a  Ground  Proximity  Warning  System 
providing  specific  voice  altitude  readouts  below  "given"  or  "pre-set" 
altitudes. 

RESPONSES: 

Pilot  #1:  Would  need  to  be  able  to  set  the  altitude(s)  in  the  cockpit. 

Pilot  #9:  A  number  of  possibilities  exist  for  usage,  however,  the 

altitudes  at  which  voice  warnings  would  be  initiated  and  the 
frequency  of  warnings  should  be  pilot  controlled. 

EX:  Operating  in  an  air-to-air  training  environment.  A 

base  altitude  of  10,000  MSL  designated  as  the  simulated 
ground  ("Kill"  floor). 


EX:  10,000  AGL  -  emergency  ejection 
2,000  AGL  -  controlled  ejection 


Pilot  #18: 


Pilot  #21: 
Pilot  #26: 

Pilot  #28: 

Pilot  #30: 
Pilot  #34: 
Pilot  #40: 

Pilot  #45: 

QUESTION  #2 

The  voice 
tails  such 
10  Launch, 

RESPONSES: 

Pilot  #2: 

Pilot  #9: 

Pilot  #14: 

Pilot  #17: 
Pilot  #31: 

Pilot  #40: 

Pilot  #41: 


)  Where  gear  are?  Up/down. 

Depends  )  Are  you  on  a  low  level? 

)  Are  you  on  a  level  radar  delivery? 

If  you  intend  to  be  low,  plus  some  voice  is  bothering  you, 
it  could  be  distracting. 

If  you  could  set  it  and  have  an  option  of  turning  it  off. 

It  would  be  useful  if  a  minimum  altitude  for  low  level  or 
ACB-1  could  be  set. 

Useful  if  able  to  shut  off  system  for  gunnery  mission  where 
it  would  be  on  continuously. 

What  about  low  levels? 

Especially  if  IFR. 

Useful  for  in  the  weather  approaches  to  landing  -  provided 
that  system  was  fully  certified  and  as  reliable  as  cockpit 
altimeter. 

Only  would  be  useful  if  it  was  cockpit  selectable. 


generator  system  could  provide  exact  threat  i nformation ^  de- 
as  the  type,  direction  and  priority  level;  for  example,  "S  A 
4  o'clock." 


Or  even  calling  a  break  turn  in  the  appropriate  direction. 

System  must  be  "fail-safe"  in  relation  to  R/T  and  must  not 
further  complicate  an  expected  comm  Jam  environment. 

In  a  threat  rich  environment,  it  could  degrade  pilot  per¬ 
formance  with  too  many  "calls." 

More  useful  if  just  speech  for  launch. 

In  a  large  strike  package  with  lots  of  UHF/VHF  transmi ssions 
in  the  target  drop,  the  voice  warning  may  be  drowned  out  or 
lost  in  all  the  excitement. 

Provided  that  voice  could  be  distinguished  from  radio  calls 
made  by  other  flights  which  would  be  using  the  same  termi¬ 
nology. 

Would  help  if  "a  sense  of  urgency"  was  present  in  the  voice. 
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QUESTION  #3 

The  voice  generator  system  could  be  used  to  give  the  pilot  reports  of 
new  threats;  for  example,  "New  guy.  Quad  two." 

RESPONSES: 


Pilot  #9: 

System  must  be  "fail-safe"  in  relation  to  R/T  and  must  not 
further  complicate  an  expected  comm  Jam  environment. 

Pilot  #14: 

If  limited  -  yes. 

Pilot  #21: 

Would  rather  have  an  "o'clock"  position. 

Pilot  #24: 

Information  would  be  especially  valuable  as  it  is  not  cur¬ 
rently  available  in  any  form. 

Pilot  #31: 

In  a  large  strike  package  with  lots  of  UHF/VHF  transmissions 
in  the  target  drop,  the  voice  warning  may  be  drowned  out  or 
lost  in  all  the  excitement. 

Pilot  #34: 

Might  be  talking  all  the  time  in  this  case  with  a  lot  of 
threats  present. 

QUESTION  #4 

The  voice 

system  could  report  the  position  and  bearing  of  a  bogey;  for 
Threat  at  two  o'clock  high." 

example,  11 

RESPONSES: 

Pilot  #9: 

System  must  be  "fail-safe"  in  relation  to  R/T  and  must  not 
further  complicate  and  expected  comm  Jam  environment. 

Pilot  #12: 

How  would  it  know  this?  If  I've  locked  T&E  “threat  at  2:00 
High"  up  on  the  radar,  then  I  already  know  where  he  is  and 
the  voice  warning  could  be  a  distraction. 

Pilot  #14: 

In  a  threat  rich  environment,  it  could  degrade  pilot  perfor¬ 
mance  with  too  many  "calls." 

Pilot  #24: 

Information  would  be  especially  valuable  as  it  is  not  cur¬ 
rently  available  in  any  form. 

Pilot  #32: 

Include  range. 

Pilot  #36: 

Current  threat  warning  systems  rely  on  a  unique  audio  to 
attract  pilot  attention,  then  require  "heads-down"  visual 
i nterpretat i on  of  a  CRT.  Straight  voice-warning  would 
eliminate  the  heads-down"  requirement  which  is  of  particular 
importance  while  flying  in  a  high  threat  area. 
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QUESTION  #5 

A  voice  message  could  inform  you  when  the  system  has  reached,  "Bingo 
fuel." 

RESPONSES: 


Pilot  #1: 

It  would  be  good  to  have  at  least  2  settings  available. 

Pilot  #9: 

Especially  in  a  task  saturated  environment,  i.e. ,  BFM,  ACM, 
high  threat  ingress/egress. 

Pilot  #21: 

Too  much  voice.  The  F- 16  HUD  flasher  is  good  enough. 

Pilot  #22: 

Present  cockpit  indications  are  sufficient;  however,  it 
would  be  nice. 

Pilot  #31: 

Not  much  different  from  current  visual  HUD  indications  that 
work  same  way. 

Pilot  #41: 

Joker  fuel  would  be  better.  Best  would  be  2  or  3  program¬ 
mable  fuel  levels. 

QUESTION  #6 

A  voice  message  could  remind  the  pilot  of  fence-check  when  20  minutes 
out. 

RESPONSES: 

Pilot  #21: 

If  you  could  set  the  times.  Actually,  2  times,  so  you  could 
get  a  reminder  both  in  and  out  (for  mode  IV,  especially). 

Pilot  #30: 

Why? 

Pilot  #31: 

Fighter  pilot  ought  to  be  able  to  handle  these  without 
something  other  than  gauges  or  warning  lights. 

Pilot  #41: 

Then  it  could  run  through  the  appropriate  systems  and  the 
pilot  could  state  what  specific  system  settings  to  be  used. 

QUESTION  #7 

The  voice 
engine  had 

generator  could  inform  the  pilot  when  the  temperature  of  the 
reached  over-heat  limits. 

RESPONSES: 

Pilot  #22: 

A  simple  "warning  warning"  and  then  appropriate  lites  would 
be  sufficient. 

Pilot  #30: 

Read  the  gauge. 
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Pilot  #31:  Fighter  pilot  ought  to  be  able  to  handle  these  without 
something  other  than  gauges  or  warning  lights. 

Pilot  #41:  The  FTIT  indicator  is  hard  to  monitor  consistently. 

QUESTION  #8 

The  voice  generator  could  detect  and  report  degradation  of  the  threat 

warning  system. 

RESPONSES: 

Pilot  #30:  Test  the  system. 

Pilot  #31:  Fighter  pilot  ought  to  be  able  to  handle  these  without 
something  other  than  gauges  or  warning  lights. 

Pilot  #40:  Only  because  most  experienced  pilots  have  learned  not  to 
rely  on  currrent  threat  warning  systems.  Assuming  that 
reliability  of  TWS  would  improve  to  the  point  that  a  pilot 
no  longer  has  to  continually  search  visually  (all  quadrants) 
for  threats,  this  would  be  very  useful. 

QUESTION  #9 

The  voice  system  could  report  an  abnormal  fuel  distribution  situation. 

RESPONSES: 

Pilot  #18:  Red  tape  on  counters  make  that  fairly  obvious. 

Pilot  #30:  Check  the  tanks. 

Pilot  #41:  Pilot  can  see  that  just  as  easily  in  F-16. 

QUESTION  #10 

A  voice  message  could  prompt  when  certain  tasks/items  should  be  per¬ 
formed,  for  example,  "Change  IFF  code." 

RESPONSES: 

Pilot  #3:  "Change  IFF  code"  -  Good. 

Fence  -  not  so  useful  -  depends  on  task. 

Pilot  #14:  Certain  functions  need  not  be  left  to  VM,  use  only  for  items 
needed  when  pilot  is  "head  out"  of  cockpit. 

Pilot  #22:  If  you  could  require  it  at  certain  times.  Additionally,  it 
cannot  take  a  lot  of  ground  time  to  set  this  in. 

Pilot  #28:  Useful  if  able  to  shut  off  system  for  gunnery  mission  where 
it  would  be  on  continuously. 
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Pilot  *30:  In  combat. 


Pilot  #41:  Especially  in  combat. 

QUESTION  #11 

The  voice  generator  could  be  linked  with  mode  changes  to  provide  addi¬ 
tional  feedback  for  such  confirmatory  changes  as  "TA/TF,  engaged,"  or 

"DP3  updated. " 

RESPONSES: 

Pilot  #21:  "Disengaged"  would  be  a  real  player,  such  as  for  the  auto 
pilot  during  nite  or  IMC  bombing. 

Pilot  #30:  "Garbaging  the  radio." 

QUESTION  #12 

The  voice  system  could  remind  the  pilot  of  critical  points  for  pull-out. 

RESPONSES: 

Pilot  #13:  In  theory  very  useful,  but  could  be  a  detriment. 

Pilot  #30:  Look  out  the  window! 

Pilot  #36:  Might  be  useful  for  preventing  self-inflicted  frag  damage 
during  live  ord.  deliveries,  i.e.,  a  "press  warning"  setup. 

Pilot  #40:  Usually,  by  the  time  the  voice  would  say  "pull  out,"  it 
would  be  too  late  for  pilot  reaction.  If  it  were  set  to  a 
higher  altitude  (for  given  A/S,  dive  angle,  A/C  gross 
weight,  configuration,  etc.),  it  would  become  a  nuisance  on 
minimum  altitude  weapons  delivery. 

QUESTION  #13 

When  requested,  the  voice  system  could  announce  the  current  ECM  mode. 

RESPONSES: 

Pilot  #3:  Voice  command  to  change  ECM  mode. 

Pilot  #9:  If  the  pilot  can  think  to  request  ECM  data,  it  only  takes  a 
quick  look  to  confirm  or  reset  the  ECM  system. 

Pilot  #21:  How?  If  by  voice,  with  a  voice  answer,  then  . 

Pilot  *30:  Look  at  the  switch. 

Pilot  #40:  What  would  be  useful  is  an  automatic  ECM  function. 
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QUESTION  #14 


The  voice  generator  could  report  hung  ordnance  information. 

RESPONSES: 

Pilot  #9:  Visual  display  already  presented  by  SMS.  Operating  in  two 
ship  elements,  a  visual  confirmation  should/would  be  accom¬ 
plished. 

Pilot  #30:  SMS  has  that  info. 

QUESTION  #15 

The  voice  generator  could  inform  the  pilot  when  the  NAV  system  has 
drifted  beyond  acceptable  limits. 

RESPONSES: 

Pilot  #30:  Look  at  the  map. 

QUESTION  #16 

The  voice  generator  could  provide  the  pilot  with  a  solution  or  descrip¬ 
tion  to  a  problem  caused  by  a  degraded  system;  for  example,  if  INS  has 
large  errors,  the  GPS  should  be  used  instead. 

RESPONSES: 

Pilot  #9:  Pilots  should  have  enough  systems  knowledge  to  analyze  a 
deficient  system  and  properly  utilize  backup  systems. 

Pilot  #14:  System  may  not  have  all  the  answers  or  big  picture  -  sug¬ 
gesting  something  to  pilot  may  be  wrong  thing  to  do. 

Pilot  #21:  The  GPS??? 

Pilot  #30:  Read  the  dash  1. 

Pilot  #41:  Often,  the  pilot  does  not  know  the  extent  of  the  degrada¬ 
tion. 

QUESTION  #17 

The  voice  system  could  verbally  enunciate  master  caution  panel  occur¬ 
rences. 

RESPONSES: 

Pilot  #30:  In  combat. 


153 


QUESTION  #18 


The  voice  generator  should  tell  the  pilot  when  the  ECM  is  locked-on, 

especially  when  the  pilot's  eyes  are  outside  the  cockpit. 

RESPONSES: 

Pilot  # 12:  ECM  doesn't  "lock-on,"  it  EMITS. 

Pilot  #13:  ?ECM  lock  on? 

QUESTION  #19 

The  voice  generator  could  inform  where  the  bogey  is  by  quadrant,  or  by 

"clock"  indications. 

RESPONSES: 

Pilot  #12:  Again,  how  does  it  know? 

Pilot  #19:  Locator  line  in  HUD  much  more  useful. 

Pilot  #21:  If  exactly  correct,  then  .  (i.e.,  left  10  o'clock, 

slightly  high). 

Pilot  #24:  Again,  the  information  itself  would  be  very  valuable  and 
presentation  by  voice  would  be  convenient. 

Pilot  #19:  Clock  position. 

Pilot  #36:  Especially  for  deep  six  threats,  etc. 

Pilot  #40:  Would  be  better  if  it  gave  bearing,  range  altitude,  and 
aspect  angle  of  a  target,  but  only  if  called  to  ao  so  by  the 
pilot. 

QUESTION  #20 

The  voice  generation  unit  could  inform  the  pilot  when  the  life  of  a 

flight  control  battery  is  getting  low,  below  some  preselected  level. 

RESPONSES: 

Pilot  #13:  Questionable.  Great  in  theory,  but  you  would  have  to  put  a 
measuring  unit  on  first.  A/C  doesn't  currently  have  one. 
Weight  on  A/C  is  a  factor.  Don't  take  away  1:1  thrust  to 
wei ght . 

Pilot  #19:  That's  what  the  light  is  for. 

Pilot  #24:  Again,  the  information  itself  would  be  very  valuable  and 
presentation  by  voice  would  be  convenient. 

Pilot  #30:  Why  not  have  a  volt  meter  with  a  water  switch? 
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Pilot  #38:  After  17  above,  a  light  or  some  video  read  out  would  be 
fi  ne. 


QUESTION  #21 

During  a  violent  spin,  the  voice  system  could  say  which  direction  the 
plane  is  turning/spinning. 


RESPONSES: 
Pilot  #1: 

Pilot  #9: 

Pilot  #12: 
Pilot  #13: 
Pilot  #21: 
Pilot  #30: 
QUESTION  *22 


The  small  number  of  times  this  might  nappen  would  not  justi¬ 
fy  the  software  space. 

Since  spins  are  not  an  everyday  occurrence,  such  an  input 
would  provide  a  positive  input  to  the  pilot  and  possibly 
preclude  a  faulty  analysis  on  his  part. 

Does  it  override  the  slow  speed  warning  horn? 

Distraction  not  required  at  that  point. 

You  would  be  too  maxed  out  to  listen. 

Look  outside! 


The  voice  system  could  enunciate  the  center  of  gravity  movement  on  an 
air  refueling  operation. 

RESPONSES: 

Pilot  #18:  Not  that  critical. 

Pilot  #30:  AOA  shows  that. 

Pilot  #40:  Only  on  A/C  with  critical  CG  movements.  Hopefully,  we're 
smart  enought  not  to  design  such  an  aircraft. 

QUESTION  #23 

Assuming  that  laser-guided  bombs  are  in  use,  speech  generation  could 
provide  warnings  to  a  pilot  that  he  was  about  to  mask  a  laser  designa¬ 
tion  with  a  particular  part  of  the  vehicle. 

RESPONSES: 

Pilot  * 3:  Not  to  present  F-16s.  May  be  useful  to  future  laser  capable 
A/C. 

Pilot  #13:  Good  idea. 

Pilot  #19:  NA. 
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Pilot  #30:  Why  not  a  visual  alert? 


QUESTION  #24 

The  deployment  of  CHAFF/FLARES  could  be  performed  by  voice  command;  for 
example,  by  speaking  "Release  CHAFF"  or  "Release  FLARES." 

RESPONSES: 

Pilot  #1:  Very  good  idea! 

Pilot  #9:  Highly  desired  capability. 

Pilot  #12:  Will  we  have  to  be  "Hot  Mic"  for  all  of  this? 

Pilot  #19:  Button  on  wall  is  much  quicker. 

Pilot  #40:  Recommend  one  word,  "CHAFF"  or  "FLARES"  or  maybe  "CHAFF  and 
FLARES." 

QUESTION  #25 

A  voice  coronand  could  be  used  to  jettison  the  stores  when  necessary;  for 
example,  a  verbal  command,  "Drop  stores." 

RESPONSES: 

Pilot  #1:  Too  large  a  possibility  of  accidental  jettison. 

Pilot  #30:  Only  for  emergency  jettison  during  take  off. 

Pilot  #40:  Again,  reconmend  one  word,  "Jettison,"  provided  that  when  I 
tell  my  wingman  on  the  radio  to  "jettison  his  ordnance,"  my 
stores  don't  all  fall  off. 

QUESTION  #26 

A  voice  command  could  be  used  to  access  the  air  terminal  information 
service  (ATIS)  in  the  spoken  form,  “ AT I S  Check." 

RESPONSES: 

Pilot  #3:  How  about  verbal  freq.  changes  (pre-set  +  manual  very  use¬ 
ful)  -  super  for  WX/nite  on  the  wing. 

Pilot  #7:  Probably  not  on  threat  environment  for  an  ATIS  check. 

Pilot  #24:  Nice,  but  hardly  critical. 

Pilot  #25:  Lazy  pilots? 

Pilot  #30:  Useless. 
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QUESTION  #27 


The  voice  command  could  be  used  to  access  JT IDS  information  as  a  spoken 
request,  "Display  JTIOS. " 

RESPONSES: 

Pilot  #30:  Useless. 

Pilot  #35:  JTIOS  -  Not  familiar  with  this  term. 

Pilot  #38:  Intentionally  left  blank. 

Pilot  #46:  What  is  JTIDS? 

QUESTION  #28 

A  voice  command  could  call  up  the  approach  plate  for  suitable  display 
access  for  various  locations;  for  example,  “Display  Nellis  Approach  Two 
One." 

RESPONSES: 

Pilot  #7:  Could  I  make  this  "extremely  very  useful,"  i.e. ,  a  rating 
greater  than  6.0? 

Pilot  #13:  How  much  programming  will  the  pilot  have  to  do  before  T.O.? 

Pilot  #21:  There  are  three  approaches  published  to  R-21  at  Nellis, 
therefore,  you  would  have  to  know  what  to  request,  thus  you 
might  have  to  look  it  up  to  know  what  to  ask  for! 

Pilot  #30:  In  weather. 

Pilot  #36:  Would  be  very  nice,  e.g.,  in  night  weather,  especially  in 
divert  situations,  etc. 

Pilot  #38:  When  I  want  to  see  an  approach  plate,  I  usually  can  afford 
to  push  a  button  rather  than  say  "Display  Nellis  Approach 
Two  One." 

Pilot  #41:  Very  useful  if  it  displayed  the  approach  rather  than  talk 
about  it. 

QUESTION  #29 

A  voice  command  could  link  the  mission  planning  and  review  information 
as  a  spoken  command,  "Mission  Review." 

RESPONSES: 

Pilot  #7:  Mission  data  card  suffices  -  too  much  flexibility  in  flight. 
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Pilot  #9:  How  is  this  data  inputed  to  the  computer  memory  with  current 
pre-taxi  computer  inputs,  INS  coordinates,  elevations, 
weapons  delivery  data,  etc.?  Additional  manual  computer 
inputs  might  require  excessive  chock  time  in  a  tactical 
situation. 

Pilot  #13:  To  what  purpose? 

Pilot  #30:  Read  the  DD175. 

QUESTION  #30 

A  voice  corumand  system  could  change  or  be  used  to  select  various  dis¬ 
plays  on  the  F-16C  MFD;  for  example,  “Display  Radar  Left." 

RESPONSES: 

Pilot  #19:  Sounds  OK,  but  I'm  not  familiar. 

QUESTION  #31 

A  voice  coronand  could  be  used  to  turn  the  ECM  POD  on;  for  example, 
“Acti vate  ACM. " 

RESPONSES: 

Pilot  #12:  ACM?  How  about  ECM? 

Pilot  #19:  Hands-on  controls  would  be  better. 

Pilot  #32:  Activate  ECM  Program  1,  Program  2,  Program  3,  etc. 

Pilot  #41:  Would  be  better  if  pilot  could  also  tell  it  which  mode  to 
select  for  ECM. 

QUESTION  #32 

A  voice  utterance  could  be  used  to  open  and  close  the  air  refueling 
doors;  for  example,  "Open  Refuel  Door." 

RESPONSES: 

Pilot  #7:  Not  in  a  threat  environment  during  air  refueling. 

Pilot  #30:  Come  on!  One  switch. 

QUESTION  #33 

A  voice  recognition  system  could  initiate  UHF/VHF  radio  channel  charges 
in  the  form,  "Select  VHF,  one-two-one-point-five." 
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RESPONSES: 


Pilot  #1:  Would  make  radio  channel  change  on  the  wing  in  the  weather 
easy. 

Pilot  #7:  I  like  this  very  much  (present  VHF  radios  in  bad  position). 
Pilot  #15:  Especially  in  weather  on  the  wing. 

Pilot  #13:  Great. 

QUESTION  #34 

A  voice  command  could  also  make  TACAN  channel  changes  by  using  a  series 
of  isolated  word  utterances;  for  example,  "Change  TACAN"  .  .  .  "eighty- 
six"  or  "Change  TACAN,  PANOCHE." 

RESPONSES: 

Pilot  #3:  Numbers  better,  nonversati le,  e.g.,  (AA  TACAN). 

Pilot  #7:  I  like  this  very  much!  (PANOCHE) 

QUESTION  #35 

A  voice  recognition  system  could  be  used  to  input  or  set  pre-flight  data 
information  concerning  route,  weapons,  fuel,  etc. 

RESPONSES: 

Pilot  #19:  What  we  need  is  a  card  reader  or  magnetic  strip  input  for 
the  FCNP. 

Pilot  #22:  Don't  know  what  this  means  -  if  you  could  input  INS  1  at/ 
longs  verbally  -  very  useful! 

Pilot  #35:  Especially  INS  lat/long's  OAP's/VRP's. 

QUESTION  #36 

Many  of  the  FCNP  keystrokes  could  be  performed  by  a  voice  recognition 
system  for  such  commands  as  the  NAV  coordinates  or  updates  by  saying, 
"Lat.  North  niner  degrees  three-two-point-eight  ..." 

RESPONSES: 

Pilot  #3:  Good  for  CAS  scenario. 

Pilot  #30:  For  changing  destination  inflight. 

Pilot  #35:  Especially  INS  lat/long's  OAP's/VRP’s. 

Pilot  #36:  Current  manual  typing  is  time  consuming  and  lends  itself  to 
input  errors.  Would  be  great  for  inflight  programming. 
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Pilot  #41:  I  think  this  would  be  a  great  asset. 


QUESTION  #37 

The  voice  recognition  system  could  be  used  to  change  any  mode  selections 
during  flight;  for  example,  "Select  Air-to-Air"  or  "Dive  Toss." 

RESPONSES: 

Pilot  #18:  How  fast  could  it  accomplish  task? 

Pilot  #20:  Depends  on  speed. 

QUESTION  #38 

The  voice  recognition  system  could  interact  with  radar  controls  to 
change  such  parameters  as  the  range,  bar  scan  pattern,  AL-EL  coverage, 
intensity,  etc. 

RESPONSES: 

Pilot  #19:  Hands-on  is  the  easiest  and  quickest  way. 

Pilot  #30:  We  can  do  it  now  with  minimal  effort. 

QUESTION  #39 

The  voice  command  system  could  be  interfaced  with  the  stores  management 
system  by  the  spoken  command,  "Select  Master  Arm." 

RESPONSES: 

Pilot  #14:  Arming  up  should  be  done  hands-on. 

Pilot  #19:  Not  sure  how  safe  it  would  be. 

Pilot  #30:  One  switch. 

QUESTION  #40 

A  verbal  command  request  could  present  the  pilot  with  the  status  of  any 
A/C  system  during  flight  (the  information  could  be  diagnostic  if  a  fault 
exists) . 

RESPONSES: 

Pilot  #3:  Emergency  procedures  display  and  commanded  action. 

Pilot  #22:  Depends  on  set-up  and  only  if  queried. 

Pilot  #35:  Diagnostic  -  Good. 


160 


i 


QUESTION  #41 

A  voice  command  could  be  used  to  enunciate  or  display  critical  items 
before  entering  FEBA;  for  example,  "Fence  Check." 

QUESTION  #42 

A  voice  command  could  be  used  to  request  aid  in  endurance  calculation  by 
the  spoken  utterance,  "Display  Endurance"  or  “Say  Endurance." 

RESPONSES: 

Pilot  #22:  Normally,  this  type  of  function  is  required  during  low  task 
operations  and  can  be  easily  accomplished  manually. 


QUESTION  #43 

A  voice  command  could  be  used  to  update  the  pilot  on  a  flyover  by  speak¬ 
ing;  for  example,  "Mark  Position." 

RESPONSES: 

Pilot  #22:  Don't  usually  use  this  function. 

QUESTION  #44 

A  voice  comnand  could  be  used  to  call  up  individual  MFD  control  pages; 
for  example,  "Stores  Management,  Left." 

RESPONSES: 

Pilot  #30:  One  switch. 

QUESTION  #45 

A  voice  command  system  could  perform  changes  in  levels  of  interior ^and 
exterior  lighting;  for  example,  by  speaking,  "Dim  Interior  Lights"  or 
"Kill  Exterior  Lights." 

RESPONSES: 

Pilot  #21:  Would  have  to  be  more  specific,  i.e.,  "position  lights, 
dim,"  “strobe  off." 


QUESTION  #46 

A  voice  command  system  could  call  up  various  checklists  and  have  the 
appropriate  one  enunciated  or  displayed;  for  example,  “Report  Air  start 
checklist"  or  "Display  Air  start  checklists." 

RESPONSES: 

Pilot  #12:  Useful  if  kept  current,  otherwise  worthless. 
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Pilot  #18:  This  is  pilot  knowledge  domain.  Many  times  this  might  be 
distracting. 

Pilot  #41:  Great,  if  up-to-date. 

Pilot  #48:  Would  be  more  useful  in  far  future  to  say  "airstart"  and  sit 
back  and  watch  it  airstart. 

QUESTION  #47 

A  voice  command  could  be  used  in  weapon  deployment;  for  example,  with  a 
verbal  "Go  slewable,"  thus  activating  target  designator  movement. 

RESPONSES: 

Pilot  #30:  One  switch. 

Pilot  #40:  Takes  too  long  in  air-to-air  combat.  In  this  case,  as  in 
other  cases  above,  the  hand  is  not  only  faster  than  the  eye 
(...it's  faster  than  the  voice). 

Pilot  #46:  F - 16  would  not  need  this! 

QUESTION  #48 

A  voice  command  could  be  interfaced  with  a  LANTIRN  system  to  activate 
delivery  modes,  weapon  selection,  and  step  from  target-to-target ;  for 
example,  “Step  target.  Pod  brighter.1' 

RESPONSES: 

Pilot  #3:  Not  F-16  present,  may  be  good  for  LANTIRN  equipped  aircraft. 

Pilot  #4:  I  do  not  know  anything  about  LANTIRN  switches  and  as  long  as 

I  am  in  a  single  seat  A/C,  I  do  not  want  to  find  out. 

Pilot  #15:  NA. 

Pilot  #20:  I  don’t  like  LANTIRN. 

Pilot  #21:  Don't  know. 

Pilot  #35:  Not  familiar  enough  with  system. 

QUESTION  #49 

Voice  command  could  be  used  to  assist  in  missile  control,  especially  in 
a  track-while-scan  mode;  for  example,  "Cage,”  "Uncage." 

RESPONSES: 

Pilot  #20:  Hands-on  is  quicker. 
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Pilot  #22:  This  is  during  extremely  high  task  saturation  and  is  now 
easily  accomplished  with  hands-on. 

Pilot  #30:  One  switch. 

QUESTION  #50 

A  voice  interactive  dialogue  capability  would  allow  the  pilot  to  ask  for 
"fuel  remaining"  by  voice  command  and  receive  the  appropriate  figure 
through  voice  generation  (this  may  be  appropirate  during  poor  night 
light  conditions). 

RESPONSES: 

Pilot  #30:  You  can  remeber  that  one,  hopefully. 

QUESTION  #51 

A  voice  interactive  system  should  allow  the  pilot  to  ask  for  "Bingo 
fuel"  and  receive  the  reply  through  speech  generation. 

QUESTION  #52 

A  voice  interactive  system  could  allow  the  pilot  to  ask  for  "fuel  home" 
and  receive  the  reply  verbally  through  speech  generation. 

QUESTION  #53 

A  voice  interactive  dialogue  system  could  be  used  to  call  up  a  future 
destination  point  (DP)  and  then  request  a  particular  ground  feature,  the 
response  to  which  may  be  verbal  in  the  form: 

Pilot:  "D.P.  three"  System  "One-tree  hill" 

Pilot:  "Distance"  System  "Two-zero  miles" 

RESPONSES: 

Pilot  #28:  Who  will  program  this  information?  What  if  you  reprogram 
destination  points  in  flight? 
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QUESTION  #54 


Assume  you  had  a  speech  I/O  system  in  an  eyes-out/hands-on-stick  and 
throttle  (HOTAS)  situation.  Please  rate  how  helpful  speech  I/O  might  be 
in  the  following  functional  areas:  (Check  one  of  the  three  blanks  on 
each  line.) 

VERY  POSSIBLY  NOT 
HELPFUL  HELPFUL  HELPFUL 


Vehicle  Control  _  _  _ 

Navigation  _  _  _ 

Fire  Control  _  _  _ 

Subsystem  Management  _  _  _ 

Threat  Management  _  _  _ 

Weapons  Delivery  _  _  _ 

Target  Acquisition  _  _  _ 

R/T  Communications  _  _  _ 

COMMENTS:  Please  comment  and  give  examples  for  as  many  of  these  eight 
categories  as  you  can.  (Use  the  back  of  the  pages  for  extra 
space. ) 


RESPONSES: 

Pilot  #3:  Vehicle  control  -  command  of  autopilot  on/off;  low  altitude 
warning;  destination  change  and  TACAN  change. 

Fire  control  -  voice  selection  of  delivery  modes  (AG)  or 
radar  and  missile  (cage/slave)  (A-A)  selections.  Threat 
management  would  be  outstanding,  especially  type  and  direc¬ 
tion  (clock)  of  threat. 

Weapons  delivery  -  may  be  good  for  action  cues  but  pilot 
still  needs  to  release  weapons. 

Target  acquisition  -  he  already  gives  good  info. 

RT  -  freq.  changes  by  voice  command. 

Pilot  #4:  Vehicle  control  -  too  many  different  situations  to  use  it 
other  than  to  engage  an  auto  pilot  system. 

Navigation  -  helpful,  but  it  is  easy  enough  now. 

Fire  control  -  great. 
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Subsystem  management  -  good  for  ECM,  CHAFF,  and  other  things 
that  are  hard  to  get  to. 

Threats  -  very  good  idea. 

Weapons  -  to  change  modes  -  great,  but  I  can  push  the  pickle 
button  or  pull  the  trigger  faster  than  I  can  speak  (more 
precise  execution  time  also). 

Target  and  R/T  -  possibly. 

Pilot  #7:  Threat  management  -  probably  highest  in  potential  in  the 
eyes-out  HOTAS  environment. 

Fire  control /weapons  delivery  -  also  high  in  potential. 

Navigation/R/T  Comm  -  especially  nice  to  have  at  night  -  in 
weather  as  a  wingman  on  the  wing.  Can  call  up  approach 
plates  and  change  frequencies  eyes-out  in  formation. 

Pilot  #3:  Nav  -  useful  as  a  back  up. 

Fire  control  -  good  in  panic  situations,  but  I'd  always  use 
the  manual  SW  as  well  as  a  voice  command  to  be  sure. 

Subsystem  -  varies  by  system  but  good  as  back-ups  or  better. 

Threat  management  -  primo,  especially  chaff/flares. 

Weapon/Tgt.  acquisition  -  good  for  changing  modes. 

R/T  -  great  for  freq.  changes  -  especially  VHF/FM. 

Pilot  #9:  All  of  the  "very  helpfuls"  reflect  upon  high  threat,  task 
saturated  situations  where  the  pilot  could  not  look  in  the 
cockpit  for  switch  changes,  i.e.,  padlocked  on  bandit  or  low 
altitude  ingress/egress  at  50  ft.  -  100  ft. 

As  for  target  acquisition,  the  system  will  help  the  pilot 
look  in  the  proper  direction,  but  he/she  must  still  see  the 
target  to  employ  ordnance. 

Pilot  #14:  Certain  situations  require  instantaneous  info  for  pilot 
(i.e.,  looks  at  airspeed  -  has  info  in  less  than  second)  to 
carry  on  a  conversation  and  get  a  verbal  response  would 
require  too  much  time  -  HOTAS  is  nice  but  not  the  panacea 
that  one  might  think. 

Pilot  #15:  Great  idea.  (Hope  it  doesn't  confuse  radio  chatter  for 
command.)  Programming  INS  is  tedious  by  hand;  could  use  it 
there. 

Pilot  #18:  Vehicle  control  -  A.C.  airspeed,  attitude,  etc.  tell  pilot 
and  he  will  make  inputs. 
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Pilot  #19:  Threat  is  important  in  that  audio  signal  gets  your  attention 
quicker  than  lights. 

Fire  control  should  be  hands-on  as  well  as  weapons  delivery. 

In  combat  or  "Hot  and  Heavy"  situations,  the  radios  must  be 
kept  clear. 

Pilot  #21:  Keep  it  simple  or  it  will  get  in  the  way.  Areas  of  small 
marginal  value  in  providing  informati an/reducing  workload 
should  be  left  out  or  the  whole  thing  could  get  out  of  hand. 

Wartime  experience  -  when  situations  get  tight,  in  critical 
emergencies,  etc.,  garbaged  up  radios  is  one  of  the  biggest 
problems.  Critical  communications  get  blocked  of  missed. 
This  would  be  like  adding  another  "radio"  and  it  needs  to 
have  "radio  disciplines"  added  with  it,  i.e.,  minimum  talk¬ 
ing  in  the  "2  mayday,  bingo,  and  lead-your-on-fire"  vein. 
Also,  inherent  should  be  the  ability  to  rapidly  turn  it  off. 

Pilot  #22:  All  the  things  you  cannot  sense  -  you  can  sense  vehicle 
motion  -  whereas  the  present  audio  information  is  not  suffi¬ 
cient  for  RHAW  systems. 

Pilot  #24:  The  system  sounds  attractive  because  of  the  nature  of  the 
information  offered  as  well  as  the  method  of  communication. 
There  are  some  situations  where  voice  comm  is  especially 
desirable  -  mostly  in  high  workload  situations  when  the 
pilot  needs  to  set-up  weapons  system  modes,  countermeasures, 
and  avionics  while  concentrating  on  outside  events. 

Pilot  #25:  Threat  -  as  stated  prior:  launches,  positions  (o’clock), 
etc. 

Weapon  -  modes,  change  modes. 

Target  acq.  -  clock,  distance,  time  to  pull  out. 

Pilot  #29:  System  would  be  helpful,  but  problems  would  be  with  inter¬ 
ference  with  other  comm  and  having  to  repeat  commands. 

Pilot  #32:  The  navigation  area  would  be  especially  good  for  the  100  ft. 

low  level  regime.  Threat  management  would  be  very  helpful 
if  the  ambiguities  can  be  worked  out.  Target  acquisition 
would  be  a  player  in  poor  visibility  or  a  camouflage  envi¬ 
ronment  . 

Pilot  #34:  NAV  -  logging  latitude/longitude  (from  forward  air  control¬ 
lers,  offsets  and  points  for  radar  work). 

Marking  points  -  updating  INS  position. 

Fire  control  -  selecting  type,  mode  of  weapons. 
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Autopilot  -  control,  parameters. 
Emergency  procedures  -  back  up. 


Pilot  #35:  Navigation  -  radar  low  level  would  really  help  when  trying 
to  break  out  of  weather  with  low  ceilings. 

Fire  Control  and  Weapons  Delivery  -  Changing  weapons  or 
delivery  mode  when  trying  to  acquire/or  after  finding  target 
that  pilot  must  remain  padlocked  to. 

R/T  -  changing  freqs.  in  weather  or  critical  phases  of 
tactical  flight  (chattermark) . 

Pilot  #36:  Vehicle  control  -  enhance  aircraft  control/ease  pilot  work¬ 
load  during  air  time  emergencies. 

Navigation  -  enhance  programming/ease  workload,  especially 
airborne  reprogramning. 

Fire  control  -  would  ease  various  mode  selection/provide 
reliability/safety  margin. 

Subsystem  Management  -  reduce  requirement  for  "heads-down" 
system  management. 

Threat  Management  -  voice  warning  for  threats  would  be  of 
immeasurable  value,  both  for  air  and  ground  threats. 

Weapons  delivery  -  relates  to  fire  control  comment  (above). 

Target  Acq.  -  any  additional  "heads-up"  cues  relating  to 
target  acquisition  would  be  helpful. 


R/T  communications  -  ease  pilot  workload,  e.g.,  radio 
changes,  etc.  Would  be  especially  good  for  emergencies,  at 
night,  in  the  weather,  etc. 

Pilot  #39:  Vehicle  control  -  I  think  most  pilots  would  agree  that  total 
aircraft  control  is  mandatory  by  the  pilot. 

Navigation,  fire  control,  target  acquisition  are  not  diffi¬ 
cult  tasks  in  the  F- 16  (physically,  the  switches  for  opera¬ 
tion  are  located  in  a  good  place). 

Threat  management  and  radio  controls  are  placed  in  less  than 
optimum  places,  thus  voice  commanded  systems,  would  be  ideal. 

Pilot  #41:  Vehicle  control  is,  of  course,  the  most  useful,  with  Nav  and 
R/T  channel  changes  also  very  useful.  The  functional  are:, 
of  threat  management  would  not  be  adapted  well  to  speech 
I/O. 

Pilot  #44:  Navigation  -  change  to  next  destination. 
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Pilot  #45:  The  system  looks  very  promising  in  the  threat  area.  Verbal¬ 
ly  informing  the  pilot  of  where  to  look  for  the  most  lethal 
threat  is  a  very  good  application. 

As  far  as  fire  control  goes,  1  would  not  like  to  see  it  go 
any  further  than  switching  delivery  modes  and  turning  on  and 
off  the  master  arm  switch. 

I  don't  like  the  idea  of  vehicle  control  through  voice 
interface. 

Pilot  #46:  An  I/O  system  could  obviously  have  benefits  in  an  eyes-out 
situation.  But,  it  must  be  closely  controlled  and  priori¬ 
tized.  You  are  busy  enough  in  an  eyes-out  situation  not  to 
be  bothered  or  distracted  by  simple  replies  or  anything  but 
threat  calls. 

Pilot  #48:  Target  acquisition  is  by  far  the  most  helpful,  since  once 
you  see  the  target  you  can  usually  strike  it. 

QUESTION  #55 


The  pilot  will  need  confirmation  that  appropriate  action  has  been  taken 
for  each  speech  command  that  he  gives.  Sometimes  the  feedback  will  be 
obvious,  as  in  a  display  mode  change.  When  feedback  is  NOT  obvious,  it 
can  be  given  either  by  speech  output  or  it  can  be  displayed  on  an  MFD, 
or  both.  Please  comment  (give  examples,  if  possible)  on  how  you  would 
like  to  get  confirmation  of  speech  command  inputs:  (Consider  (I)  speech 
output,  (2)  displayed  information,  and  (3)  both.) 

RESPONSES: 


Pilot  #1:  Speech  output  would  be  the  best  bet,  i.e.,  when  asking  for 

fuel ,  TACAN  channel  change. 


Pilot  #2:  The  use  of  a  response,  i.e.,  "Received"  would  be  good. 

Pilot  #3:  A  side  tone  on  acceptance  of  the  command  along  with  even 

obvious  display  changes.  You're  not  always  looking  inside 
when  you  want  a  change  or  action  made. 

Pilot  #4:  I  like  what  you  said  earlier.  Have  it  talk  back,  but  have 

the  capability  to  override  it,  or  turn  it  off  -  possibly  it 

could  just  give  a  beep  if  it  did  what  you  said,  or  maybe  say 
"Done." 


Pilot  #5:  Minimize  the  speech  output.  In  air-to-ground  modes,  i.e., 
NAV  -  A-G.  Provide  feedback  on  Hud.  Give  verbal  feedback 
on  Hud. 


Allow  pilot  to  select  which  priorities  he  wants  to  hear. 
Pilot  #7:  Speech  output  primary,  but  a  I  ways  confirmed  visually. 
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Pilot  #8:  Must  primarily  be  info  display  in  life  or  killing  situations 
(weapons  deliv.  threat,  etc.),  but  speech  output  okay  for 
routine/admin,  items. 

Pilot  #9:  (3)  Both.  Again,  the  tactical  situation  and  the  degree  of 

task  saturation  would  dictate  the  capability  to  read  a 
display.  A  voice  (primary)  display  (secondary)  would  be 
most  advantageous. 

Pilot  #10:  I  think  a  tone  could  be  used  to  confirm  the  actions  are 
complete.  It  would  be  short,  very  easy  to  hear  and  iden¬ 
tify,  and  the  same  for  all  commands. 

Pilot  #12:  Probably  both,  but  with  a  speech  override  so  as  not  to 
conflict  with  UHF/VHF.  Displayed  for  5  sec.  or  50  and  then 
display  back  to  norm. 

Pilot  #14;  MFO  is  preferable  because  it  could  be  confirmed  quicker  - 
taking  less  of  pilot's  ;ime  to  digest  conversation. 

Pilot  #15:  Displayed  information  and  audio  acceptance  tone,  i.e.,  beep. 

Pilot  #16:  Beep  in  headset. 

Pilot  #17;  Some  speech.  Some  displayed.  Depends  on  tne  situation. 

Pilot  #18:  Speech  is  best,  but  I  would  want  3  display  also. 

Pilot  #19:  It  would  have  to  be  both,  just  in  case. 

Pilot  #20:  NA. 

Pilot  #2i :  Speech  "read  back"  if  your  command  after  action  complete 
would  be  a  good  way.  "Display  so  and  so"  -  action  -  "so  anc 
so  displayed." 

Pilot  #22:  Could  be  a  tone  -  or  could  De  a  repetition  of  tne  comana. 

Pilot  #25:  Repeat  command  or  short  code  that  it  is  understood. 

Pilot  #26:  Chaff/Flares  (speech  output  -  "Chaff /FI ares  releasee",. 

Pilot  #28:  Speech  output. 

Pilot  #29:  Would  like  option  to  have  either  one. 

Pilot  #30:  I  don't  want  this  thing  garbaging  up  communications. 

Pilot  #31:  System  could,  in  most  cases,  report  part  or  all  of  ycur 
command  when  it  has  completed  it.  The  visual  indications, 
such  as  a  change  in  HUD  symbology,  would  confirm  that  your 
command  had  been  accomplished  also.  I  think  if  you  could 
use  both  in  as  many  cases  as  possible,  it  would  be  better. 
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Pilot  #32:  Or  you  could  simply  use  a  single  tone  generation  to  give 
confirmation. 

Pilot  #34:  Unsure  of  confirmation  methods. 

Pilot  #35:  To  much  chatter  with  speech  output. 

Pilot  #36:  Speech  output  (as  well  as  input)  must  be  kept  to  an  absolute 
minimum  so  as  not  to  conflict  with  flight  comm.,  e.g.,  an 
easily  recognizable  tone,  confirmed  with  a  Hud  visual  cue 
would  seem  adequate. 

Pilot  #37:  (3)  Some  inputs  need  immediate  feedback,  such  as  flare 

dispensing  or  ECM  activation  since  the  tactics  could  change 
i f  they  fail. 

Pilot  #33:  Both. 

Pilot  #39:  A  1/2  second  medium  range  tone  or  "Beep"  would  be  sufficient 
to  confirm  commended  actions  have  taken  place;  in  conjuntion 
with  a  display  MFD.  A  high  pitched  4KHZ  Beep  could  be  used 
as  noting  commanded  action  is  not  understood,  cannot  be 
compl ied  with ,  etc. 

Pilot  #41:  How  about  just  a  certain  frequency  tone  or  beep  to  acknow¬ 
ledge  request  (i.e.,  keep  time  to  a  min.)  and  then  another 
when  appropriate  action  is  taken. 

Pilot  #44:  If  not  speech  for  feedback,  a  small  indicator  light  (symbol 
on  HUD)  to  let  you  know  the  information  has  been  computed 
and  is  ready  to  be  read  on  an  MFD. 

Pilot  ?4o:  Never  speech  output  in  a  high  threat  arena,  it  would  be 
bothersome.  However,  it  would  be  helpful  in  safe  territory. 
Maybe  the  confirmation  mode  could  change  with  the  fence. 

Pilot  #47:  Speech  output. 

Pilot  #43:  In  most  situation,  I  would  like  to  actually  hear  that  it  is 
completed,  since  my  eyes  must  be  out  of  cockpit;  otherwise, 
I  wouldn't  need  the  voice  system  in  the  first  place. 

QUESTION  #56 

As  previously  indicated,  R/T  will  have  priority  on  the  audio  channel. 

One  suggested  method  for  prioritizing  the  speech  system  outputs  is  as 

follows: 

Priority  #1  -  Ultra  important  message  which  assists  in  the  safe  conduct 
of  the  mission.  These  messages  will  override  any  other 
message  being  transmitted  by  the  speech  generation  system. 
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Priority  #2  -  Very  important  messages  which  assist  in  the  management  of 
the  extraordinary  events  experienced  during  a  mission, 
e.g.,  flight  control  system  servo  failure. 

Priority  #3  -  Important  messages  which  assist  in  the  management  of 
ordinary  events  experienced  in  flight,  e.g.,  endurance/ 
fuel  remaining. 

Priority  #4  -  Routine  messages  which  assist  the  pilot  in  routine  work, 
e.g.,  reciting  checklists. 

Each  Priority  # 2  through  #4  speech  message  will  be  completed  before  the 
next  message  is  commenced,  unless  a  priority  number  one  message  inter¬ 
venes.  When  more  than  one  message  is  in  the  queue,  the  priority  will  be 
adjusted  before  the  next  message  is  transmitted.  Incomplete  messages 
will  be  reassigned  to  their  correct  priority  position  and  retransmitted 
as  complete  messages.  Messages  which  are  delayed  more  than  20  seconds 
by  the  priority  rules  will  not  be  transmitted  and  will  be  cancelled. 

Do  you  agree  with  this  priority  concept?  Yes  _  No  _ 

RESPONSES: 

Pilot  #3:  3  and  4  should  be  changed.  Combat  related  items  should  have 

higher  priority  than  ordinary  flight  items. 

Pilot  #7 :  Recormend  a  combat-oriented  priori t i zat ion  be  made  different 

than  a  non-combat-oriented  situation.  For  example,  "chaff- 
flare  dispense"  may  step  to  priority  one  in  combat. 

Pilot  #8:  The  20  second  rule  must  allow  some  info  on  what  isn't  being 
transmitted.  Also,  some  combat  situations  will  change  the 
priorities  from  a  peace  time  mission,  e.g.,  chaff/flares  vs. 
some  warnings  of  syst.  malfunctions. 

Pilot  #9:  A  pause  mode  to  allow  for  ai rcraft-to-ai rcraft  communica¬ 

tions  is  required.  Ai rcraft-to- ai rcraft  or  aircraft-to- 
ground  communications  should  actuate  the  pause  mode  for  any 
and  all  computer  generated  messages. 

Pilot  #10:  Any  radio  transmission  should  have  the  highest  priority  anc 
override  all  I/O  transmissions. 

Pilot  #14:  In  high  stress  situations,  only  ultra  messages  should  get 
through. 

Pilot  #13:  Realize  in  combat  and  critical  situation  everything  gets 
real  busy  and  the  voice  might  be  a  nuisance. 

Pilot  #21:  Keep  it  to  three. 

Pilot  #25:  Sounds  great  -  Flash  (Gordon)  never  had  it  so  good. 


Pilot  #30: 

Pilot  #31: 

Pilot  #36: 

Pilot  #37 : 

Pilot  #41: 
Pilot  #42: 

Pilot  #44: 

Pilot  #46: 

Pilot  #43: 

QUESTION  #57 

Please  add 
cations  in 

RESPONSES: 

Pilot  #3: 

Pilot  #4: 

Pilot  #5: 


Too  much  going  on  at  once.  In  the  F-16,  when  there  is  an 
electrical  problem,  there  are  usually  quite  a  few.  Results 
and  failures  don't  need  to  be  verbally  informed,  one  after 
the  other. 

The  one  place  I  can  see  this  thing  going  crazy  is  in  the 
target  area  in  a  big  fight  when  there  are  lots  of  threats, 
radio  chatter,  and  all  kinds  of  things  going  on  at  once. 
The  speech  system,  other  aircraft  warning  systems,  and  all 
the  radio  activity  may  make  things  almost  impossible. 

Will  this  priority  be  determined  by  the  individual  pilot/ 
geared  to  specific  mission  requirements? 

I  would  not  cancel  any  messages  unless  computer  memory  space 
is  a  problem. 

Make  the  20  secs  a  variable  dependent  on  message. 

Potential  to  add  to  pilot  workload  during  emergencies,  etc. 
Oon't  need  any  more  lights,  bonks,  whistles,  etc. 

Still  need  a  manual  cut  out  in  case  of  difficulties.  I 
still  want  control  over  the  system.  In  emergencies,  the 
improper  display  or  annoying  speech  could  be  very  dis¬ 
tracting. 

It  will  probably  take  a  lot  of  refining  after  inputs  from 
actual  application. 

Since  things  may  happen  all  at  once,  there  needs  to  be  some 
pri ority. 


any  other  comments,  general  or  specific,  on  speech  I/O  appli- 
future  tactical  aircraft. 


Voice  command  will  add  a  new  and  better  dimension  to  new 
generation  “heads  up"  fighters.  Looking  forward  to  it. 

Flying  "hot  mic"  bothers  me.  I  would  like  to  see  it  so  I 
could  talk  to  the  system  "cold  mic"  so  I  wouldn't  have  to 
listen  to  myself  breathe,  grunt,  etc. 

Speech  I/O,  in  my  opinion,  has  good  applications  in  a  very 
limited  area.  In  general,  this  area  is  when  the  pilot  must 
have  his  eyes  outside  and  keep  them  outside,  i.e.,  air-to- 
air.  It  is  otherwise  redundant  at  best  and  a  possible 
distraction  at  worse.  Speech  activated  switches  could  not 
replace  HOTAS  switches. 
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Pilot  #7:  I  would  like  most  of  the  features  you  mentioned.  I  would 
like  to  select  which  modes  I  want  to  use  and  I  would  like  to 
select  audio  or  visual  confirmation  for  each  mode. 

I  would  also  like  to  tell  the  audio  confirm  to  shut  itself 
off  or  turn  itself  on. 

Pilot  #8:  I'm  glad  to  see  pilots  being  consulted  fairly  early  on. 

Engineers/designers  (unless  also  users)  cannot  fully  compre¬ 
hend  the  requirements  for  such  a  system  that  result  from 
psychological  and  personality  oriented  factors. 

Pilot  #9:  What  will  be  the  impact  of  pilot  pitch  and  tone  changes 
during  highly  critical  situations?  Will  the  computer  voice 
recogni ze? 

What  "fail-safe"  measure  will  be  incorporated  to  assure 
phonetic  sampling  by  the  computer  will  not  key  it  to  respond 
to  a  sound-alike  in  its  vocabulary? 

EX:  Two  place  aircraft: 

Pi lot-to-Navigator  (WSO):  "Cheer  Up." 

Computer  I/O  System:  "Roger,  Gear  Up." 

Pilot  #12:  How  long  will  it  take  to  program  all  this  stuff  in?  Can  it 
be  "read"  into  memory  with  a  computer  tape?  Can  it  recog¬ 
nize  voice  under  the  stress  of  high  Gs?  How  can  we  keep  it 
current  (checklists,  approach  plates,  etc.)?  We  would  need 
an  on/off  switch  in  case  it  malfunctions.  Will  we  have  to 
be  "Hot  Mic"  all  the  time  to  talk  to  the  system?  S„.ne, 
non-time  critical  items  need  a  confirmation  message  prior  to 
implementation  (examples,  pilot  says  “nott  corrs,"  system 
asks  "drop  stores?"  Yes/No.  Pilot  can  clear  system  with  a 
panicky  "NO!" 

Pilot  #14:  Possibilities  are  endless  -  however,  let's  not  get  into  cn^ 
"new  toy"  mode  with  speech  I/O.  Let's  make  sure  we  don': 
overload  pilot  with  messages. 

Pilot  #16:  Real  pilots  are  supposed  to  be  able  to  fly  their  ^ets  witn- 
out  the  help  of  some  clown  talking  in  their  ear.  Apout  cr  ■ 
only  good  thing  are  freq.  changes. 

Pilot  #13:  From  a  safety  viewpoint: 

1)  Refer  this  project  to  aero  psychologists. 

2)  In  many  phases  of  flight,  this  could  be  as  much  a  dis¬ 
traction  as  AID. 

3)  Leaves  a  lot  of  room  for  pilot  error  anu  reliance  on  a 
system  that  is  doing  some  critical  jobs  too. 
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4)  Will  system  tell  pilot  if  it  is  malfunctioning?  Can 
pilot  turn  it  off? 

5)  Pilot  must  have  positive  feedback  that  system  has  accom¬ 
plished  task  it  was  commanded  to  perform. 

6)  This  sounds  like  a  Weapon  System  Officer  (WSO)  in  ab- 
stensia!  From  "Star  Wars:" 

a)  There  will  always  be  a  happy  hour! 

b)  Dive  toss  will  never  work! 

c)  There  will  always  be  a  WSO! 

Pilot  #21:  Cosmic,  but  has  obvious  value  in  a  single  place  aircraft. 
Looks  like  an  idea  whose  time  is  coming. 

Pilot  #23:  Good  ideas!  Do  it. 

Pilot  #25:  Possibilities  limitless. 

Pilot  #28:  Don't  take  too  many  tasks  away  from  the  pilot.  It  may  be 
more  helpful  if  you  concentrate  on  threat  warning,  such  as 
saying  what  is  on  the  RHAW  when  the  pilot  is  looking  out¬ 
side. 

Pilot  #29:  Sounds  good  on  paper,  but  won't  work  until  the  system  can 
understand  commands  from  a  voice  under  stress/pitch  change/ 
abbreviated  speech,  etc. 

Pilot  #30:  Of  every  fault,  I  can  read  on  the  panel  instantly. 

Pilot  #31:  How  much  or  how  little  this  thing  can  do  for  the  pilot  and 
whom  will  depend  on  individual  pilot's  situation  awareness 
(SA).  Fighter  pilots  use  this  as  a  gauge  of  how  well  you 
understand  what  is  going  on  in  the  total  environment  around 
you.  Your  SA  in  a  multi  bogey  ACM  engagement  or  a  high 
threat  target  area  is  what  will  keep  you  alive,  and  it  is 
not  hard  to  get  maxed  out  quickly.  SA  is  a  function  of 
experience,  training,  and,  many  times,  just  plain  natural 
ability.  The  speech  system  may  be  very  helpful  in  lots  of 
instances  like  emergencies  or  an  instrument  approach  in  bad 
weather.  Where  it  can  offer  valuable  assistance  is  when  the 
pilot  is  not  maxed  out  and  can  use  its  information.  Max  out 
his  SA  and  put  him  in  a  position  where  he  is  fighting  to 
stay  alive  and  pretty  quickly  the  amount  of  information  he 
can  absorb  and  use  reaches  the  limit  of  his  personal  SA 
capability.  From  then  on,  the  overloaded  SA  system  now 
degenerates  rapidly  as  he  is  saturated.  The  speech  system 
can  then  be  more  harmful  than  good,  and  he's  got  to  use  the 
"shut-up  switch."  How  long  each  pilot  can  make  use  of  the 
system  is  going  to  vary  considerably.  How  you  determine 
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what  information  the  system  provides  should  also  be  depen¬ 
dent  on  this  "usefulness  level."  I  think  you  are  on  the 
right  track  in  providing  a  "shut-up"  feature  because  the 
system  can  reach  a  point  where  it  detracts  rather  than  adds 
to  pilot  performance. 

Pilot  #34:  Initiating  built-in  tests  for  maintenance. 

Pilot  #36:  Pilots  must  have  the  ultimate  control  of  such  systems,  i.e., 
a  basic  ON-OFF  switch.  There  are  numerous  critical  inflight 
situations  which  could  only  be  made  worse  by  untimely  voice 
info.  The  concept  seems  good  and  could  go  a  long  way  in 
reducing  mundane  pilot  workloads,  e.g.,  typing  waypoint 
coordinates,  offsets,  frequency  changes,  SMS  programming, 
etc.  This,  in  turn,  enhances  flight  safety/mi ssion  accom¬ 
plishment. 

Pilot  #39:  I'm  concerned  about  too  much  chatter.  Excess  chatter  could 
do  more  harm  then  good.  Menial  tasks  in  a  relaxed  environ¬ 
ment  could  be  done  manually  (i.e.,  "Open  air  refuel  door," 
"Request  range  data"). 

Some  commands  could  be  combined,  such  as  "Perform  Fence 
Check."  The  steps  to  be  performed  could  be  programmed  prior 
to  flight. 

Functions  which  are  difficult  or  cumbersome  to  perform  in  a 
particular  jet  would  be  especially  adept  to  be  performed 
verbally  (in  the  F- 16  maybe  FCNP  operations,  CHAFF  &  FLARES, 
ECM,  etc.  any  heads-down  operation).  Functions  that  a  pilot 
may  have  to  perform  when  looking  over  his  shoulder  or  that 
require  extensive  heads-down  time  would  oe  ideal  items  for 
verbal  commands.  The  fire  control  operation  in  the  F- 16  is 
perfectly  satisfactory;  it  can  be  operated  while  looking 
over  the  shoulder. 

Pilot  #40:  As  I  see  it,  the  biggest  drawback  to  this  concept  is  noise 
saturation  in  the  cockpit.  In  air-to-air  combat  arena, 
pilot  has  already  reached  the  saturation  point  (about  15 
yrs.  ago)  with  inner/inter  flight  UHF  and  VHF  communica¬ 
tions,  missile  tones,  RWR  audio,  AOA  and  slow  speed  tones, 
and  enemy  noise  jamming.  Most  pilots  learned  to  totally 
tune  out  some  of  these  "helpful"  noises  such  as  RWR  and  ATT 
tones. 

Second  problem  -  most  pilots  have  difficulty  talking  (commu¬ 
nicating)  effectively  to  other  flight  members  during  a  hot 
and  heavy  air  battle.  To  ask  them  also  to  talk  to  thei^ 
airplane  during  such  times  would  be  too  much.  Under  the 
optional  "use  manual  switches  or  voice  command"  concent 
training  pay  back  would  be  reduced,  i.e..  you  would  have  tc 
train  a  pilot  to  use  the  manual  switches  like  today's  train¬ 
ing  and  add  in  training  for  voice  command  (like  training  an 
F-4  pilot  and  navigator  to  operate  as  a  crew). 
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Concept  has  some  value  in  areas  out  of  the  irrenediate  combat 
arena.  In  fact,  in  some  cases,  I  think  it  would  be  very 
valuable,  but  only  if  scrubbed  by  a  team  of  combat-ready 
fighter  pilots. 

Pilot  #42:  Sounds  like  you're  over-designing  a  system.  Suggest  using 
only  for  small  number  of  critical  items.  Don't  use  for 
anything  that  can  be  controlled  on  the  stick  or  throttle. 

Pilot  #48:  Would  be  excellent  if  recognizer  could  distinguish  voice 
when  screaming  as  well  as  talking,  since  I  will  probably  be 
screaming  in  stress  situations. 

MISCELLANEOUS  COMMENTS: 

Pilot  #7:  General  comment  on  1st  page  states:  Part  1  comments  - 
threat  warning/life  threats  are  paramount  and  you  have 
indicated  all  these  areas. 

Comments  on  Part  II  cover  age  state:  I  liked  most  of  the 
items  in  this  section.  A  lot  of  them  are  "nice-to-haves"  - 
others  are  "gotta-haves."  It's  incredible...  the  areas 
you've  looked  at  are  areas  I  have  had  thoughts  about  for 
some  time.  You've  hit  the  primary  goal  on  the  head...  pilot 
workload  reduction  in  a  task  saturated  environment. 

Pilot  #8:  Comments  on  1st  page:  Personal  Opinion  -  adding  another 
sensory  demand  (voice)  to  those  already  used  (propriocep¬ 
tive,  visual,  etc.)  may  have  a  negative  net  efficiency  in 
some  cases. 

Pilot  #30:  I  believe  anything  that  lessens  pilot  workload  and  prevents 
task  saturation  will  be  helpful,  but  the  aircraft  reliabili¬ 
ty  seems  to  be  sacrificed  as  a  result.  The  kids  who  fix  up 
and  maintain  the  systems  are  still  underpaid  and  underedu¬ 
cated.  Will  the  new  "improvements'1  really  be  that  reliable 
and  capable  of  "front-line  maintenance"  and  quick  fixes? 

Pilot  #38:  I  am  extremely  interested  by  the  concept. 

LIST  OF  PILOTS  WITHOUT  ANY  COMMENTS: 

Pilot  #6;  Pilot  #11;  Pilot  #13;  Pilot  #27;  Pilot  #33;  and  Pilot  #43. 
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